Please see Development Guide for project level development instructions.

This guide is targeted at developers looking to contribute to Feast components in the main Feast repository:

• Feast Python SDK / CLI
• Feast Java Serving
• Feast Go Client

A quick list of things to keep in mind as you're making changes:

• As you make changes
  • Make your changes in a forked repo (instead of making a branch on the main Feast repo)
  • Sign your commits as you go (to avoid DCO checks failing)
  • Rebase from master instead of using git pull on your PR branch
  • Install pre-commit hooks to ensure all the default linters / formatters are run when you push.
• When you make the PR
  • Make a pull request from the forked repo you made
  • Ensure you add a GitHub label (i.e. a kind tag to the PR (e.g. kind/bug or kind/housekeeping)) or else checks will fail.
  • Ensure you leave a release note for any user facing changes in the PR. There is a field automatically generated in the PR request. You can write NONE in that field if there are no user facing changes.
  • Please run tests locally before submitting a PR (e.g. for Python, the local integration tests)
  • Try to keep PRs smaller. This makes them easier to review.

Fork the Feast Github repo and clone your fork locally. Then make changes to a local branch to the fork.

See Creating a pull request from a fork

Setup pre-commit to automatically lint and format the codebase on commit:

1. Ensure that you have Python (3.7 and above) with pip, installed.
2. Install pre-commit with pip & install pre-push hooks

pip install pre-commit
pre-commit install --hook-type pre-commit --hook-type pre-push

3. On push, the pre-commit hook will run. This runs make format and make lint.

⚠️ Warning: using the default integrations with IDEs like VSCode or IntelliJ will not sign commits. When you submit a PR, you'll have to re-sign commits to pass the DCO check.

Use git signoffs to sign your commits. See https://docs.github.com/en/github/authenticating-to-github/managing-commit-signature-verification for details

Then, you can sign off commits with the -s flag:

git commit -s -m "My first commit"

GPG-signing commits with -S is optional.

Our preference is the use of git rebase [master] instead of git merge : git pull -r.

Note that this means if you are midway through working through a PR and rebase, you'll have to force push: git push --force-with-lease origin [branch name]

Setting up your development environment for Feast Python SDK / CLI:

1. Ensure that you have Docker installed in your environment. Docker is used to provision service dependencies during testing.
2. Ensure that you have make, Python (3.7 and above) with pip, installed.
3. Recommended: Create a virtual environment to isolate development dependencies to be installed

# create & activate a virtual environment
python -m venv venv/
source venv/bin/activate

3. Upgrade pip if outdated

pip install --upgrade pip

4. Install development dependencies for Feast Python SDK / CLI

pip install -e "sdk/python[dev]"

Feast Python SDK / CLI codebase:

• Conforms to Black code style
• Has type annotations as enforced by mypy
• Has imports sorted by isort
• Is lintable by flake8

To ensure your Python code conforms to Feast Python code standards:

• Autoformat your code to conform to the code style:

make format-python

• Lint your Python code before submitting it for review:

make lint-python

Setup pre-commit hooks to automatically format and lint on commit.

Unit tests (pytest) for the Feast Python SDK / CLI can run as follows:

make test-python

⚠️ Local configuration can interfere with Unit tests and cause them to fail:

• Ensure no AWS configuration is present and no AWS credentials can be accessed by boto3
• Ensure Feast Python SDK / CLI is not configured with configuration overrides (ie ~/.feast/config should be empty).

There are two sets of tests you can run:

1. Local integration tests (for faster development)
2. Full integration tests (requires cloud environment setups)

To get local integration tests running, you'll need to have Redis setup:

Redis

1. Install Redis: Quickstart
2. Run redis-server

Now run make test-python-universal-local

To test across clouds, on top of setting up Redis, you also need GCP / AWS / Snowflake setup.

Note: you can manually control what tests are run today by inspecting RepoConfiguration and commenting out tests that are added to DEFAULT_FULL_REPO_CONFIGS

GCP

1. Install the Cloud SDK.
2. Then run login to gcloud:

gcloud auth login
gcloud auth application-default login

3. Export GCLOUD_PROJECT=[your project] to your .zshrc

AWS

1. TODO(adchia): flesh out setting up AWS login (or create helper script)
2. Modify RedshiftDataSourceCreator to use your credentials

Snowflake

• See https://signup.snowflake.com/

Then run make test-python-integration. Note that for Snowflake / GCP / AWS, this will create new temporary tables / datasets.

See Java contributing guide

Setting up your development environment for Feast Go SDK:

• Install Golang, protoc with the Golang & grpc plugins

Build the Feast Go Client with the go toolchain:

go build

Feast Go Client codebase:

• Conforms to the code style enforced by go fmt.
• Is lintable by go vet.

Autoformat your Go code to satisfy the Code Style standard:

go fmt

Lint your Go code:

go vet

Setup pre-commit hooks to automatically format and lint on commit.

Unit tests for the Feast Go Client can be run as follows:

go test

• Update your current master on your forked branch and make a pull request against your own forked master.
• Enable workflows by going to actions and clicking Enable Workflows.
  • Pushes will now run your edited workflow yaml file against your test code.
  • Unfortunately, in order to test any github workflow changes, you must push the code to the branch and see the output in the actions tab.

• pr-integration-tests workflow is skipped
  • Add ok-to-test github label.
• pr-integration-tests errors out with Error: fatal: invalid refspec '+refs/pull//merge:refs/remotes/pull//merge'
  • This is because github actions cannot pull the branch version for some reason so just find your PR number in your pull request header and hard code it into the uses: actions/checkout@v2 section (i.e replace refs/pull/${{ github.event.pull_request.number }}/merge with refs/pull/<pr number>/merge)
• AWS/GCP workflow
  • Currently still cannot test GCP/AWS workflow without setting up secrets in a forked repository.