This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On demand feature views allows data scientists to use existing features and request time data (features only available
at request time) to transform and create new features at the time the data is read from the online store. Users define
python transformation logic which is executed in both historical retrieval and online retrieval paths.
On Demand Feature Views (ODFVs) allow data scientists to use existing features and request-time data (features only
available at request time) to transform and create new features. Users define Python transformation logic which is
executed during both historical retrieval and online retrieval. Additionally, ODFVs provide flexibility in
applying transformations either during data ingestion (at write time) or during feature retrieval (at read time),
controlled via the `write_to_online_store` parameter.
By setting `write_to_online_store=True`, transformations are applied during data ingestion, and the transformed
features are stored in the online store. This can improve online feature retrieval performance by reducing computation
during reads. Conversely, if `write_to_online_store=False` (the default if omitted), transformations are applied during
feature retrieval.
### Why use on demand feature views?
This enables data scientists to easily impact the online feature retrieval path. For example, a data scientist could
1. Call `get_historical_features` to generate a training dataframe
2. Iterate in notebook on feature engineering in Pandas/Python
3. Copy transformation logic into on demand feature views and commit to a dev branch of the feature repository
3. Copy transformation logic into ODFVs and commit to a development branch of the feature repository
4. Verify with `get_historical_features` (on a small dataset) that the transformation gives expected output over historical data
5. Verify with `get_online_features` on dev branch that the transformation correctly outputs online features
6. Submit a pull request to the staging / prod branches which impact production traffic
5. Decide whether to apply the transformation on writes or on reads by setting the `write_to_online_store` parameter accordingly.
6. Verify with `get_online_features` on dev branch that the transformation correctly outputs online features
7. Submit a pull request to the staging / prod branches which impact production traffic
## CLI
Expand All
@@ -32,10 +40,18 @@ See [https://github.com/feast-dev/on-demand-feature-views-demo](https://github.c
### **Registering transformations**
On Demand Transformations support transformations using Pandas and native Python. Note, Native Python is much faster but not yet tested for offline retrieval.
On Demand Transformations support transformations using Pandas and native Python. Note, Native Python is much faster
but not yet tested for offline retrieval.
When defining an ODFV, you can control when the transformation is applied using the write_to_online_store parameter:
- `write_to_online_store=True`: The transformation is applied during data ingestion (on write), and the transformed features are stored in the online store.
- `write_to_online_store=False` (default when omitted): The transformation is applied during feature retrieval (on read).
We register `RequestSource` inputs and the transform in `on_demand_feature_view`:
df['conv_rate_adjusted'] = features_df['conv_rate'] * 1.1 # Adjust conv_rate by 10%
return df
```
Then to ingest the data with the new feature view make sure to include all of the input features required for the
transformations:
```python
from feast import FeatureStore
import pandas as pd
store = FeatureStore(repo_path=".")
# Data to ingest
data = pd.DataFrame({
"driver_id": [1001],
"event_timestamp": [pd.Timestamp.now()],
"conv_rate": [0.5],
"acc_rate": [0.8],
"avg_daily_trips": [10],
})
# Ingest data to the online store
store.push("driver_hourly_stats_view", data)
```
### **Feature retrieval**
{% hint style="info" %}
Expand Down
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: Adding documentation for On Demand Feature Transformations with writes #4607
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
feat: Adding documentation for On Demand Feature Transformations with writes #4607
Filter by extension
Viewed files
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing