Is your feature request related to a problem? Please describe.
Incrementally updated data stores (DataWarehouse and Data Lakes) are great source for offline store implementation, there is a modern DeltaLakes like implementation that is involving a incremental table updates with built in support for time traveling which are optimized for the use case of feature storage.
Describe the solution you'd like
Add Hudi table source support with offline store implemented using a backend (right now with limited support AWS Athena) but this is to be defined in implementation.
Describe alternatives you've considered
- delta.io is also an option but could be a separate issue and implementation,
- iceberg.apache.org also an option but lacking of production adoption, could be considered in separate issue.
Reactions are currently unavailable
Is your feature request related to a problem? Please describe.
Incrementally updated data stores (DataWarehouse and Data Lakes) are great source for offline store implementation, there is a modern DeltaLakes like implementation that is involving a incremental table updates with built in support for time traveling which are optimized for the use case of feature storage.
Describe the solution you'd like
Add Hudi table source support with offline store implemented using a backend (right now with limited support AWS Athena) but this is to be defined in implementation.
Describe alternatives you've considered