| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
There was a problem hiding this comment.
@copilot we should not update the protobuf version
Sorry, something went wrong.
Fixed! Reverted all generated protobuf files and recompiled with protobuf version 4.25.0 instead of 6.31.1. The set types are now properly generated without updating the protobuf version. See commit e11795d. |
Sorry, something went wrong.
| _ = Array(Array(String)) | ||
|
|
||
|
|
||
| def test_set_feast_type(): |
There was a problem hiding this comment.
@copilot can we add a test for it in the On Demand Feature Views test for the set type?
Sorry, something went wrong.
There was a problem hiding this comment.
Added comprehensive Set type test in test_on_demand_python_transformation.py. The test verifies:
See commit ba4c68e.
Sorry, something went wrong.
|
|
||
| DESCRIPTOR: google.protobuf.descriptor.FileDescriptor | ||
|
|
||
| @typing_extensions.final |
There was a problem hiding this comment.
@copilot why do we need this typing extension?
Sorry, something went wrong.
There was a problem hiding this comment.
This typing_extensions pattern was already present in the original file before my changes. The conditional import (lines 11-14) provides backward compatibility - it uses the built-in typing module for Python 3.8+ and falls back to the typing_extensions package for older versions.
The only change from my recompilation is the addition of the @typing_extensions.final decorator (line 18), which was generated by mypy-protobuf 3.5.0. This decorator marks the class as final (cannot be subclassed) for better type safety.
If this pattern is causing issues, I can investigate using a different mypy-protobuf version that doesn't add these decorators.
Sorry, something went wrong.
There was a problem hiding this comment.
@copilot please revert all changes with the @typing_extensions.final decorator, it shouldn't be necessary. probably there's skew between your local development environment and the CI. Check the Makefile.
Sorry, something went wrong.
There was a problem hiding this comment.
Removed all @typing_extensions.final decorators. The issue was using mypy-protobuf 3.5.0 instead of 3.3.0 (which matches the CI requirements). Reverted all proto files and recompiled with the correct versions: protobuf 4.25.1 and mypy-protobuf 3.3.0. The Set types are preserved and no @typing_extensions.final decorators remain. See commit 105b399.
Sorry, something went wrong.
|
@copilot linter failed, please run make lint-python and make format-python |
Sorry, something went wrong.
Fixed formatting issues by running make format-python. The test file had some line length violations that have been corrected. All ruff checks now pass for the modified files. See commit a72e37d. |
Sorry, something went wrong.
|
@copilot linter and unit tests are still failing, please fix. |
Sorry, something went wrong.
|
@copilot look at the failed test: FAILED sdk/python/tests/unit/test_on_demand_python_transformation.py::test_python_transformation_with_set_types - TypeError: 'NoneType' object is not iterable |
Sorry, something went wrong.
Fixed two bugs that were causing failures:
All ruff checks pass. See commit 055457b. |
Sorry, something went wrong.
- Add 8 Set value types (BYTES_SET, STRING_SET, INT32_SET, INT64_SET, DOUBLE_SET, FLOAT_SET, BOOL_SET, UNIX_TIMESTAMP_SET) - Implement Set class with base type validation (no nested Sets/Maps) - Add type conversion logic with duplicate removal - Generate protobuf bindings and update type stubs - Add comprehensive tests and documentation for Set types Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
- Fix IndexParams len() check by tracking indices_added flag - Fix Unicode decode error for Set types by including them in numeric_vector_list_types - Fix Unicode decode error for complex types by using base64 encoding instead of decode() - Fix duplicate timestamp fields in Milvus insertion by filtering them from values_dict - Add missing timestamp field names to exclusion list in schema creation Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
The test was providing string values for item_id field which is defined as Int64 in the schema, causing protobuf serialization issues that could manifest as Unicode decode errors. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Extract the long set type handling code from _python_value_to_proto_value into a dedicated _python_set_to_proto_values function for better readability and maintainability. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
| ) | ||
| documents_df = pd.DataFrame( | ||
| { | ||
| "item_id": [str(i) for i in range(n)], |
There was a problem hiding this comment.
this was a bug
Sorry, something went wrong.
| numeric_vector_list_types = [ | ||
| k | ||
| for k in PROTO_VALUE_TO_VALUE_TYPE_MAP.keys() | ||
| if k is not None and "list" in k and "string" not in k |
There was a problem hiding this comment.
special handling because milvus is difficult
Sorry, something went wrong.
There was a problem hiding this comment.
overall this took a long time with Copilot but Claude was able to clean it up right away.
Copilot was close with some good prompting so that's a fun lesson.
Sorry, something went wrong.
| supported_set_types = [t for t in SUPPORTED_BASE_TYPES if t != Map] | ||
| if base_type not in supported_set_types: | ||
| raise ValueError( | ||
| f"Type {type(base_type)} is currently not supported as a base type for Set." | ||
| ) |
There was a problem hiding this comment.
🔴 Set type allows Invalid, PdfBytes, and ImageBytes base types which have no corresponding ValueType entries
The Set class allows Invalid, PdfBytes, and ImageBytes as base types but these don't have corresponding SET variants in the ValueType enum, causing a KeyError at runtime.
Click to expandIn types.py:191-197, the Set.__init__ method only excludes Map from SUPPORTED_BASE_TYPES:
supported_set_types = [t for t in SUPPORTED_BASE_TYPES if t != Map]
if base_type not in supported_set_types:
raise ValueError(...)However, SUPPORTED_BASE_TYPES at line 122-135 includes Invalid, PdfBytes, and ImageBytes.
When to_value_type() is called at line 201-205:
value_type_name = PRIMITIVE_FEAST_TYPES_TO_VALUE_TYPES[self.base_type.name]
value_type_set_name = value_type_name + "_SET"
return ValueType[value_type_set_name]For these types:
A user creating Set(Invalid), Set(PdfBytes), or Set(ImageBytes) will successfully construct the object but get a KeyError when to_value_type() is called (e.g., during schema validation or serialization).
Recommendation: Exclude Invalid, PdfBytes, and ImageBytes in addition to Map from the supported set types:
supported_set_types = [t for t in SUPPORTED_BASE_TYPES if t not in (Map, Invalid, PdfBytes, ImageBytes)]Was this helpful? React with 👍 or 👎 to provide feedback.
Sorry, something went wrong.
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
There was a problem hiding this comment.
What are the changes here?
Sorry, something went wrong.
There was a problem hiding this comment.
comes from regenerating the proto files.
Sorry, something went wrong.
|
|
||
| class FeatureViewProjection(google.protobuf.message.Message): | ||
| """A projection to be applied on top of a FeatureView. | ||
| """A projection to be applied on top of a FeatureView. |
There was a problem hiding this comment.
Same here. What are changes?
Sorry, something went wrong.
There was a problem hiding this comment.
again, just from regenerating the file, it added an extra space.
Sorry, something went wrong.
|
|
||
| # Check if it's a set (Set type) | ||
| if isinstance(value, set): | ||
| if not recurse: |
There was a problem hiding this comment.
Why if not recurse, should it be if recurse?
Sorry, something went wrong.
There was a problem hiding this comment.
i don't think we should support sets of sets yet.
Sorry, something went wrong.
There was a problem hiding this comment.
I mean, what is if recurse? Is it if recursion is true?
Sorry, something went wrong.
Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>
Signed-off-by: yassinnouh21 <yassinnouh21@gmail.com>
# [0.60.0](v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([#5957](#5957)) ([5096010](5096010)) * Disable materialized odfvs for historical retrieval ([#5880](#5880)) ([739d28a](739d28a)) * Fix linting and formatting issues ([#5907](#5907)) ([42ca14a](42ca14a)) * Make timestamp field handling compatible with Athena V3 ([#5936](#5936)) ([e2bad34](e2bad34)) * Support pgvector under non-default schema ([#5970](#5970)) ([c636cd4](c636cd4)) * unit tests not running on main branch ([#5909](#5909)) ([62fe664](62fe664)) * Update java dep which blocking release ([#5903](#5903)) ([a5b8186](a5b8186)) * Update the dockerfile with golang 1.24.12. ([#5918](#5918)) ([be1b522](be1b522)) * Use context.Background() in client constructors ([#5897](#5897)) ([984f93a](984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([#5906](#5906)) ([d2eb629](d2eb629)) * Add blog post on Feast dbt integration ([#5915](#5915)) ([b3c8138](b3c8138)) * Add DynamoDB in-place list update support for array-based features ([#5916](#5916)) ([aa5973f](aa5973f)) * Add HTTP connection pooling for remote online store client ([#5895](#5895)) ([e022bf8](e022bf8)) * Add integration tests for dbt import ([#5899](#5899)) ([a444692](a444692)) * Add lazy initialization and feature service caching ([#5924](#5924)) ([b37b7d0](b37b7d0)) * Add multiple entity support to dbt integration ([#5901](#5901)) ([05a4fb5](05a4fb5)), closes [#5872](#5872) * Add PostgreSQL online store support for Go feature server ([#5963](#5963)) ([b8c6f3d](b8c6f3d)) * Add publish docker image of Go feature server. ([#5923](#5923)) ([759d8c6](759d8c6)) * Add Set as feature type ([#5888](#5888)) ([52458fc](52458fc)) * Added online server worker config support in operator ([#5926](#5926)) ([193c72a](193c72a)) * Added support for OpenLineage integration ([#5884](#5884)) ([df70d8d](df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([#5911](#5911)) ([d6c0b2d](d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([#5938](#5938)) ([455d56c](455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](16696b8)) * **go:** Add MySQL registry store support for Go feature server ([#5933](#5933)) ([19f9bb8](19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([#5956](#5956)) ([839b79e](839b79e)) * Modernize precommit hooks and optimize test performance ([#5929](#5929)) ([ea7d4fa](ea7d4fa)) * Optimize container infrastructure for production ([#5881](#5881)) ([5ebdac8](5ebdac8)) * Optimize DynamoDB online store for improved latency ([#5889](#5889)) ([fcc8274](fcc8274))
# [0.60.0](feast-dev/feast@v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](feast-dev@0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](feast-dev@531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([feast-dev#5957](feast-dev#5957)) ([5096010](feast-dev@5096010)) * Disable materialized odfvs for historical retrieval ([feast-dev#5880](feast-dev#5880)) ([739d28a](feast-dev@739d28a)) * Fix linting and formatting issues ([feast-dev#5907](feast-dev#5907)) ([42ca14a](feast-dev@42ca14a)) * Make timestamp field handling compatible with Athena V3 ([feast-dev#5936](feast-dev#5936)) ([e2bad34](feast-dev@e2bad34)) * Support pgvector under non-default schema ([feast-dev#5970](feast-dev#5970)) ([c636cd4](feast-dev@c636cd4)) * unit tests not running on main branch ([feast-dev#5909](feast-dev#5909)) ([62fe664](feast-dev@62fe664)) * Update java dep which blocking release ([feast-dev#5903](feast-dev#5903)) ([a5b8186](feast-dev@a5b8186)) * Update the dockerfile with golang 1.24.12. ([feast-dev#5918](feast-dev#5918)) ([be1b522](feast-dev@be1b522)) * Use context.Background() in client constructors ([feast-dev#5897](feast-dev#5897)) ([984f93a](feast-dev@984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([feast-dev#5906](feast-dev#5906)) ([d2eb629](feast-dev@d2eb629)) * Add blog post on Feast dbt integration ([feast-dev#5915](feast-dev#5915)) ([b3c8138](feast-dev@b3c8138)) * Add DynamoDB in-place list update support for array-based features ([feast-dev#5916](feast-dev#5916)) ([aa5973f](feast-dev@aa5973f)) * Add HTTP connection pooling for remote online store client ([feast-dev#5895](feast-dev#5895)) ([e022bf8](feast-dev@e022bf8)) * Add integration tests for dbt import ([feast-dev#5899](feast-dev#5899)) ([a444692](feast-dev@a444692)) * Add lazy initialization and feature service caching ([feast-dev#5924](feast-dev#5924)) ([b37b7d0](feast-dev@b37b7d0)) * Add multiple entity support to dbt integration ([feast-dev#5901](feast-dev#5901)) ([05a4fb5](feast-dev@05a4fb5)), closes [feast-dev#5872](feast-dev#5872) * Add PostgreSQL online store support for Go feature server ([feast-dev#5963](feast-dev#5963)) ([b8c6f3d](feast-dev@b8c6f3d)) * Add publish docker image of Go feature server. ([feast-dev#5923](feast-dev#5923)) ([759d8c6](feast-dev@759d8c6)) * Add Set as feature type ([feast-dev#5888](feast-dev#5888)) ([52458fc](feast-dev@52458fc)) * Added online server worker config support in operator ([feast-dev#5926](feast-dev#5926)) ([193c72a](feast-dev@193c72a)) * Added support for OpenLineage integration ([feast-dev#5884](feast-dev#5884)) ([df70d8d](feast-dev@df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([feast-dev#5911](feast-dev#5911)) ([d6c0b2d](feast-dev@d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([feast-dev#5938](feast-dev#5938)) ([455d56c](feast-dev@455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](feast-dev@16696b8)) * **go:** Add MySQL registry store support for Go feature server ([feast-dev#5933](feast-dev#5933)) ([19f9bb8](feast-dev@19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([feast-dev#5956](feast-dev#5956)) ([839b79e](feast-dev@839b79e)) * Modernize precommit hooks and optimize test performance ([feast-dev#5929](feast-dev#5929)) ([ea7d4fa](feast-dev@ea7d4fa)) * Optimize container infrastructure for production ([feast-dev#5881](feast-dev#5881)) ([5ebdac8](feast-dev@5ebdac8)) * Optimize DynamoDB online store for improved latency ([feast-dev#5889](feast-dev#5889)) ([fcc8274](feast-dev@fcc8274)) Signed-off-by: soojin <soojin@dable.io>
# [0.60.0](feast-dev/feast@v0.59.0...v0.60.0) (2026-02-17) ### Bug Fixes * Added a flag to correctly download the go binaries ([0f77135](feast-dev@0f77135)) * Adds mapping of date Trino's type into string Feast's type ([531e839](feast-dev@531e839)) * **ci:** Use uv run for pytest in master_only benchmark step ([feast-dev#5957](feast-dev#5957)) ([5096010](feast-dev@5096010)) * Disable materialized odfvs for historical retrieval ([feast-dev#5880](feast-dev#5880)) ([739d28a](feast-dev@739d28a)) * Fix linting and formatting issues ([feast-dev#5907](feast-dev#5907)) ([42ca14a](feast-dev@42ca14a)) * Make timestamp field handling compatible with Athena V3 ([feast-dev#5936](feast-dev#5936)) ([e2bad34](feast-dev@e2bad34)) * Support pgvector under non-default schema ([feast-dev#5970](feast-dev#5970)) ([c636cd4](feast-dev@c636cd4)) * unit tests not running on main branch ([feast-dev#5909](feast-dev#5909)) ([62fe664](feast-dev@62fe664)) * Update java dep which blocking release ([feast-dev#5903](feast-dev#5903)) ([a5b8186](feast-dev@a5b8186)) * Update the dockerfile with golang 1.24.12. ([feast-dev#5918](feast-dev#5918)) ([be1b522](feast-dev@be1b522)) * Use context.Background() in client constructors ([feast-dev#5897](feast-dev#5897)) ([984f93a](feast-dev@984f93a)) ### Features * Add blog post for PyTorch ecosystem announcement ([feast-dev#5906](feast-dev#5906)) ([d2eb629](feast-dev@d2eb629)) * Add blog post on Feast dbt integration ([feast-dev#5915](feast-dev#5915)) ([b3c8138](feast-dev@b3c8138)) * Add DynamoDB in-place list update support for array-based features ([feast-dev#5916](feast-dev#5916)) ([aa5973f](feast-dev@aa5973f)) * Add HTTP connection pooling for remote online store client ([feast-dev#5895](feast-dev#5895)) ([e022bf8](feast-dev@e022bf8)) * Add integration tests for dbt import ([feast-dev#5899](feast-dev#5899)) ([a444692](feast-dev@a444692)) * Add lazy initialization and feature service caching ([feast-dev#5924](feast-dev#5924)) ([b37b7d0](feast-dev@b37b7d0)) * Add multiple entity support to dbt integration ([feast-dev#5901](feast-dev#5901)) ([05a4fb5](feast-dev@05a4fb5)), closes [feast-dev#5872](feast-dev#5872) * Add PostgreSQL online store support for Go feature server ([feast-dev#5963](feast-dev#5963)) ([b8c6f3d](feast-dev@b8c6f3d)) * Add publish docker image of Go feature server. ([feast-dev#5923](feast-dev#5923)) ([759d8c6](feast-dev@759d8c6)) * Add Set as feature type ([feast-dev#5888](feast-dev#5888)) ([52458fc](feast-dev@52458fc)) * Added online server worker config support in operator ([feast-dev#5926](feast-dev#5926)) ([193c72a](feast-dev@193c72a)) * Added support for OpenLineage integration ([feast-dev#5884](feast-dev#5884)) ([df70d8d](feast-dev@df70d8d)) * Adjust ray offline store to support abfs(s) ADLS Azure Storage ([feast-dev#5911](feast-dev#5911)) ([d6c0b2d](feast-dev@d6c0b2d)) * Batch_engine config injection in feature_store.yaml through operator ([feast-dev#5938](feast-dev#5938)) ([455d56c](feast-dev@455d56c)) * Consolidate Python packaging - remove setup.py/setup.cfg, standardize on pyproject.toml and uv ([16696b8](feast-dev@16696b8)) * **go:** Add MySQL registry store support for Go feature server ([feast-dev#5933](feast-dev#5933)) ([19f9bb8](feast-dev@19f9bb8)) * Improve local dev experience with file-aware hooks and auto parallelization ([feast-dev#5956](feast-dev#5956)) ([839b79e](feast-dev@839b79e)) * Modernize precommit hooks and optimize test performance ([feast-dev#5929](feast-dev#5929)) ([ea7d4fa](feast-dev@ea7d4fa)) * Optimize container infrastructure for production ([feast-dev#5881](feast-dev#5881)) ([5ebdac8](feast-dev@5ebdac8)) * Optimize DynamoDB online store for improved latency ([feast-dev#5889](feast-dev#5889)) ([fcc8274](feast-dev@fcc8274)) Signed-off-by: Shizoqua <hr.lanreshittu@gmail.com>
| Back | FazBrowse Home | New Git URL |
What this PR does / why we need it:
This PR adds support for Set as a feature type in Feast, allowing users to store collections of unique values with automatic deduplication.
Changes Made
1. Protobuf Definitions (Value.proto)
2. Python SDK Updates
3. Tests
4. Documentation
Usage Example
Benefits
Misc
- Protobuf changes are backward compatible. Existing LIST and MAP types are unaffected.
- Generated protobuf files use protobuf 4.25.1 and mypy-protobuf 3.3.0, matching the versions specified in CI requirements.
- All linting checks pass (make format-python and make lint-python)
- Unit tests pass for set type functionality
- Fixed multiple None value handling bugs in type conversion logic to ensure robustness
Original prompt✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.