Problem
Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved datasets. The current adapter still loads projected data into pandas and performs joins locally. Backfill execution and connecting its output to Feast remain manual.
ML platform teams should be able to use Feast's historical retrieval APIs with Chronon-backed computation at a scale that does not require the full source dataset in the Feast client's memory.
Proposed solution
Implement this in stages:
- Define an execution contract mapping a Feast feature selection to a versioned Chronon configuration, entity keys/timestamps, execution target, and output location. Distinguish reading an existing materialization from explicitly requesting a new backfill.
- Add an optional backend that submits or attaches to a job through a supported Chronon execution integration. Expose job status, failures, timeout/cancellation behavior, and output location through the retrieval workflow.
- Support registering or persisting remotely produced results without downloading the entire dataset. Make local dataframe collection explicit, and reject unsupported transformations instead of silently falling back to unbounded local execution.
Keep the existing local Parquet path compatible. Chronon retains ownership of feature computation and job execution; registration alone must not launch jobs.
Acceptance criteria
- A runnable integration example executes a real Chronon backfill and consumes its output through get_historical_features() and Feast dataset APIs.
- The execution contract supports a documented remote entity-input format and records configuration/input versions, job identity, and output provenance.
- Regression tests preserve documented point-in-time, TTL, timestamp, field-mapping, feature-name, missing-value, and entity-row semantics; unsupported combinations fail clearly.
- A reproducible workload exceeding the configured client memory budget completes through the distributed path without loading the full source or result into pandas.
- Retries do not duplicate completed work or expose partial outputs; job failure, timeout, and cancellation behavior is tested.
- Document backend setup, credentials, execution ownership, supported on-demand transformations, operational limits, and migration from the current workflow.
Alternatives considered
Continue externally orchestrating Chronon and manually publishing Parquet for local retrieval, or hand-wire Chronon outputs to another Feast offline store. These remain useful options, but leave users to maintain the handoff and execution semantics.
Context
Follow-up to #6188; related definition/metadata work: #6982.
Starting points: Chronon offline adapter, current historical retrieval documentation, and retrieval regression tests.
Backend choice and the submission API need a design proposal before implementation.
Problem
Feast already supports point-in-time retrieval over Chronon Parquet outputs, including TTL, created-timestamp tie-breaking, column projection, local on-demand transformations, and saved datasets. The current adapter still loads projected data into pandas and performs joins locally. Backfill execution and connecting its output to Feast remain manual.
ML platform teams should be able to use Feast's historical retrieval APIs with Chronon-backed computation at a scale that does not require the full source dataset in the Feast client's memory.
Proposed solution
Implement this in stages:
Keep the existing local Parquet path compatible. Chronon retains ownership of feature computation and job execution; registration alone must not launch jobs.
Acceptance criteria
Alternatives considered
Continue externally orchestrating Chronon and manually publishing Parquet for local retrieval, or hand-wire Chronon outputs to another Feast offline store. These remain useful options, but leave users to maintain the handoff and execution semantics.
Context
Follow-up to #6188; related definition/metadata work: #6982.
Starting points: Chronon offline adapter, current historical retrieval documentation, and retrieval regression tests.
Backend choice and the submission API need a design proposal before implementation.