feat: Add integration tests for dbt import with local dbt project by Copilot · Pull Request #5877 · feast-dev/feast · GitHub

What this PR does / why we need it:

Adds comprehensive integration tests for the dbt import feature (PR #5827), which previously lacked end-to-end testing with actual dbt project setup and Feast CLI integration.

Changes

Test Infrastructure

Created sdk/python/tests/integration/dbt/ with 600+ lines covering:
- Manifest parsing with dbt-artifacts-parser
- Tag filtering (feast, ml, recommendations)
- Model selection by name
- All 3 data source types (BigQuery, Snowflake, File)
- Type mapping validation (INT32/64, FLOAT32/64, STRING, TIMESTAMP)
- FeatureView generation with schema validation
- Column exclusion (entity, timestamp, custom)
- Code generation workflow

Test dbt Project

Minimal dbt project in test_dbt_project/ with 3 models:
- driver_features: INT types, multiple tags
- customer_features: STRING entity
- product_features: FLOAT32, tag filtering test
Seed data (CSV files) for all three models to enable actual dbt execution
DuckDB-only profile configuration for local testing without external service dependencies
Models use ref() to reference seeds, enabling full dbt workflow testing
Manifest generation: Removed pre-committed manifest.json; CI now generates it via dbt build to ensure tests validate actual dbt output

CI/CD - End-to-End Workflow

GitHub Actions workflow dbt-integration-tests.yml
Runs on Python 3.11 & 3.12
Triggers on dbt code/test changes
Uses Makefile for dependency installation (make install-python-dependencies-ci)
Executes full dbt workflow: dbt deps, dbt build, dbt test with seed data
Tests Feast CLI integration:
- Creates Feast project with SQLite online store configuration
- Runs feast dbt import command with generated manifest
- Verifies Feast objects (entities, feature views) are created correctly
Validates both dbt compilation and Feast import work together

Bug Fixes

Removed deprecated PytestUnhandledCoroutineWarning from pytest.ini
Fixed test failures by ensuring manifest.json is generated by dbt during CI rather than pre-committed

Test Example

def test_get_models_with_tag_filter(self, parser):
    """Test filtering models by dbt tag."""
    # Filter by 'ml' tag
    ml_models = parser.get_models(tag_filter="ml")
    assert len(ml_models) == 2
    
    # Filter by 'recommendations' tag
    rec_models = parser.get_models(tag_filter="recommendations")
    assert len(rec_models) == 1

Which issue(s) this PR fixes:

Related to #3335

Misc

Comprehensive documentation added for test structure and dbt project. The workflow now provides complete end-to-end testing from dbt model definition through to Feast feature store integration, including actual data materialization to SQLite online store. The manifest.json is dynamically generated during CI to ensure tests validate real dbt compilation output.

Original prompt

This section details on the original issue you should resolve

<issue_title>Add integration tests for dbt import with local dbt project</issue_title>
<issue_description>## Context
PR #5827 added dbt integration but lacks proper integration tests with an actual dbt project setup.

Description

As mentioned by @franciscojavierarceo in PR #5827, we need a lightweight local dbt project that:

Sets up a minimal dbt project structure

Runs dbt compile to generate manifest.json

Tests the full feast dbt import workflow end-to-end

Validates generated Feast objects

Acceptance Criteria

Create test dbt project in sdk/python/tests/integration/dbt/

Add test that compiles dbt project and imports to Feast

Test all three data source types (bigquery, snowflake, file)

Verify FeatureViews are created correctly

Test tag filtering and model selection

Related

PR feat: Add dbt integration for importing models as FeatureViews #5827

Issue Feast <> dbt integration #3335</issue_description>

<agent_instructions>please setup dbt in a github action integration test and test the dbt parser that was implemented in PR #5827</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes Add integration tests for dbt import with local dbt project #5869

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

@YassinNouh21 mind reviewing?

@copilot we should use the Makefile to install dependencies here

Updated to use make install-python-dependencies-ci in commit 621cefb.

@copilot we should noly test with the sqlite setup, no need for us to use bigquery or snowflake imo

Removed BigQuery and Snowflake profiles, kept only DuckDB setup in commit e25d08f.

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

@copillot we should also run the standard dbt commands; i.e,.

dbt run
dbt build
dbt test

after that we should then test the new feast cli and test that feast materialization works from a dbt model into a sqlite online store.

Added dbt commands (deps, build, test) and Feast CLI testing with SQLite online store in commit ba38da0. The workflow now:

Installs dbt-core and dbt-duckdb
Runs dbt build and test with seed data
Creates a Feast project with SQLite online store
Tests feast dbt import command
Verifies Feast objects are created correctly

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

See inline comments for specific fixes needed.

fix those issues copilot

Found several API mismatches that will cause test failures

entity.join_keys should be entity.join_key - Entity uses singular string, not a list

@copilot please revise

feature_view.entities[0] contains entity names as strings, not Entity objects. Should be: assert feature_view.entities[0] == entity.name

@copilot please revise

Incorrect count. With 3 models you get: 3 DataSources + 3 Entities + 3 FeatureViews = 9 objects total. Should be: assert len(all_objects) == 9

@copilot please revise

@franciscojavierarceo Found some issues in this PR that need attention:

Critical bugs (will cause test failures):

Line 230: entity.join_keys → should be entity.join_key (singular)
Line 254: feature_view.entities[0] stores entity names as strings, not Entity objects
Line 527: Object count should be 9, not 12 (3 entities + 3 sources + 3 views)

Architecture concerns:

These are labeled "integration tests" but don't actually run dbt or use a real database - they're unit tests testing the parser against a static manifest
Manifest uses outdated v9 schema instead of v12
Missing error handling & edge case coverage

The inline comments on lines 191, 215, 488 show the exact fixes needed for the critical bugs. Happy to help if you need clarification on any of these.

@YassinNouh21 you have to tag @copilot in every comment for it to update the code 😅

@franciscojavierarceo I thought he will continue working on it 😂

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

@copilot tests are failing

…generate it Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

@copilot tests are failing

Fixed in commit 586f825. The issue was an incorrectly pre-committed manifest.json from unit tests (with wrong project name "sample_dbt_project" instead of "feast_integration_test"). The manifest is now generated fresh by dbt build in the CI workflow before tests run.

Initial plan

c3f2e0f

Copilot AI assigned Copilot and franciscojavierarceo Jan 16, 2026

Copilot started work on behalf of franciscojavierarceo January 16, 2026 20:15 View session

Copilot AI and others added 3 commits January 16, 2026 20:25

Add dbt integration tests and test dbt project

9bee2c7

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Add manifest.json and GitHub Actions workflow for dbt tests

8ba2205

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Add documentation and README files for dbt integration tests

3432eb5

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI changed the title [WIP] Add integration tests for dbt import with local dbt project feat: Add integration tests for dbt import with local dbt project Jan 16, 2026

Copilot AI requested a review from franciscojavierarceo January 16, 2026 20:31

Copilot finished work on behalf of franciscojavierarceo January 16, 2026 20:31

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Copilot started work on behalf of franciscojavierarceo January 17, 2026 04:15 View session

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Use Makefile for dependency installation in dbt workflow

621cefb

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI requested a review from franciscojavierarceo January 17, 2026 04:17

Copilot finished work on behalf of franciscojavierarceo January 17, 2026 04:17

Copilot started work on behalf of franciscojavierarceo January 17, 2026 04:18 View session

franciscojavierarceo reviewed Jan 17, 2026

View reviewed changes

Simplify profiles.yml to only use DuckDB for testing

e25d08f

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot AI requested a review from franciscojavierarceo January 17, 2026 04:20

Copilot finished work on behalf of franciscojavierarceo January 17, 2026 04:20

YassinNouh21 reviewed Jan 18, 2026

View reviewed changes

YassinNouh21 suggested changes Jan 18, 2026

View reviewed changes

Copilot started work on behalf of franciscojavierarceo January 21, 2026 16:14 View session

Add dbt commands and Feast CLI integration testing with SQLite

ba38da0

Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot finished work on behalf of franciscojavierarceo January 21, 2026 16:19

Copilot started work on behalf of franciscojavierarceo January 21, 2026 16:19 View session

Copilot finished work on behalf of franciscojavierarceo January 21, 2026 16:20

Copilot started work on behalf of franciscojavierarceo January 22, 2026 18:24 View session

Fix test failures by removing pre-committed manifest and letting dbt …

586f825

…generate it Co-authored-by: franciscojavierarceo <4163062+franciscojavierarceo@users.noreply.github.com>

Copilot finished work on behalf of franciscojavierarceo January 22, 2026 18:28

+                      with:
+                        enable-cache: true
+                    - name: Install dependencies

+                    path: ':memory:'
+                    threads: 1
+              bigquery_test:

+                        uv pip install --system -e "."
+                        uv pip install --system dbt-artifacts-parser pytest pytest-xdist
+                    - name: Run dbt integration tests

+                  def test_create_feature_view(self, parser):
+                      """Test creating Feast FeatureView from dbt model."""
+                      mapper = DbtToFeastMapper(data_source_type="bigquery")

+                      # Check that schema excludes entity and timestamp columns
+                      feature_names = {f.name for f in feature_view.schema}
+                      assert "driver_id" not in feature_names  # Entity column excluded

+                  def test_code_generation_workflow(self, parser):
+                      """Test workflow that generates Python code."""
+                      models = parser.get_models(model_names=["driver_features"])

feat: Add integration tests for dbt import with local dbt project by Copilot · Pull Request #5877 · feast-dev/feast · GitHub

Uh oh!

feat: Add integration tests for dbt import with local dbt project - #5877

Conversation

Copilot AI commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Changes

Test Example

Which issue(s) this PR fixes:

Misc

Description

Acceptance Criteria

Related

Comments on the Issue (you are @copilot in this section)

Uh oh!

franciscojavierarceo commented Jan 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

YassinNouh21 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YassinNouh21 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YassinNouh21 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YassinNouh21 commented Jan 18, 2026

Uh oh!

franciscojavierarceo commented Jan 19, 2026

Uh oh!

YassinNouh21 commented Jan 19, 2026

Uh oh!

franciscojavierarceo commented Jan 22, 2026

Uh oh!

Copilot AI commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jan 16, 2026 •
edited
Loading

franciscojavierarceo Jan 17, 2026 •
edited
Loading

YassinNouh21 left a comment •
edited
Loading

YassinNouh21 left a comment •
edited
Loading