FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix: Read Kafka and Kinesis sources without a batch source from proto by LuisFigueroaG · Pull Request #6949 · feast-dev/feast · GitHub

Repository navigation

fix: Read Kafka and Kinesis sources without a batch source from proto - #6949

Merged
ntkathole merged 1 commit into
feast-dev:masterfrom
LuisFigueroaG:fix/stream-source-optional-batch-source
Oct 7, 2026
Merged

ntkathole merged 1 commit into
feast-dev:masterfrom
LuisFigueroaG:fix/stream-source-optional-batch-source

Conversation

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

batch_source is optional on KafkaSource and KinesisSource, but their from_proto checked it with if data_source.batch_source. An unset proto sub-message is still truthy, so the empty message was passed to DataSource.from_proto and raised:

ValueError: Could not identify the source type being added.

So a stream source applied without a batch source could be written to the registry but not read back (get_data_source, list_data_sources, etc.):

k = KafkaSource(
    name="k",
    timestamp_field="ts",
    message_format=JsonFormat(schema_json="a int"),
    kafka_bootstrap_servers="b:9092",
    topic="t",
)
DataSource.from_proto(k.to_proto())  # ValueError

This switches both checks to HasField("batch_source"), the same check PushSource.from_proto already uses.

Which issue(s) this PR fixes:

Same failure as #3852, which was closed without a fix.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Added a parametrized round-trip test for Kafka and Kinesis sources without a batch source; it fails on master and passes with this change. ruff format --check, ruff check and mypy pass on the changed files, and the unit suite passes apart from tests that need torch or a Java runtime, which fail the same way on master in my environment.

LuisFigueroaG requested a review from a team as a code owner October 6, 2026 00:23

haoxu0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

lgtm

codecov-commenter commented Oct 7, 2026 •
edited
Loading

Copy link
Copy Markdown

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.07%. Comparing base (211ecb8) to head (f907c5f).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #6949   +/-   ##
=======================================
  Coverage   49.07%   49.07%           
=======================================
  Files         433      433           
  Lines       54330    54330           
  Branches     7917     7917           
=======================================
+ Hits        26663    26664    +1     
  Misses      25788    25788           
+ Partials     1879     1878    -1     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 50.47% <ø> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/data_source.py 83.06% <ø> (+0.65%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 211ecb8...f907c5f. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

batch_source is optional on KafkaSource and KinesisSource, but from_proto
checked it with a truthiness test. An unset proto sub-message is still
truthy, so the empty message was parsed as a data source and raised
"Could not identify the source type being added.", which made these
sources impossible to read back from the registry.

Use HasField("batch_source"), as PushSource.from_proto already does.

Signed-off-by: LuisFigueroaG <luis.h.figueroa.g@gmail.com>
ntkathole force-pushed the fix/stream-source-optional-batch-source branch from f907c5f to ad50bc7 Compare October 7, 2026 04:46
ntkathole merged commit a620f5c into feast-dev:master Oct 7, 2026
2 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants


Back | FazBrowse Home | New Git URL