Commit 82ae109
feat: Improve local dev experience with file-aware hooks and auto parallelization
- Make precommit hooks file-aware: format and lint only changed Python files
- Add conditional template hook that only runs when template files change
- Update test-python-integration-local to use -n auto and --dist loadscope
- Add new pytest markers: slow, cloud, local_only for better test selection
- Add format-python-files and lint-python-files Makefile targets
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent e2bad34 commit 82ae109
3 files changed
Lines changed: 38 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -3,21 +3,29 @@ default_stages: [commit] | |||
| 3 | 3 | repos: | |
| 4 | 4 | - repo: local | |
| 5 | 5 | hooks: | |
| 6 | - - id: format | ||
| 7 | - name: Format | ||
| 6 | + # File-aware format hook - only runs on changed Python files | ||
| 7 | + - id: format-files | ||
| 8 | + name: Format Changed Files | ||
| 8 | 9 | stages: [commit] | |
| 9 | 10 | language: system | |
| 10 | - entry: make format-python | ||
| 11 | - pass_filenames: false | ||
| 12 | - - id: lint | ||
| 13 | - name: Lint | ||
| 11 | + types: [python] | ||
| 12 | + entry: uv run ruff check --fix | ||
| 13 | + pass_filenames: true | ||
| 14 | + | ||
| 15 | + # File-aware lint hook - only runs on changed Python files | ||
| 16 | + - id: lint-files | ||
| 17 | + name: Lint Changed Files | ||
| 14 | 18 | stages: [commit] | |
| 15 | 19 | language: system | |
| 16 | - entry: make lint-python | ||
| 17 | - pass_filenames: false | ||
| 20 | + types: [python] | ||
| 21 | + entry: uv run ruff check | ||
| 22 | + pass_filenames: true | ||
| 23 | + | ||
| 24 | + # Conditional template hook - only runs when template files change | ||
| 18 | 25 | - id: template | |
| 19 | 26 | name: Build Templates | |
| 20 | 27 | stages: [commit] | |
| 21 | 28 | language: system | |
| 29 | + files: ^infra/templates/|\.jinja2$ | ||
| 22 | 30 | entry: make build-templates | |
| 23 | 31 | pass_filenames: false | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -58,11 +58,29 @@ format-python: ## Format Python code | |||
| 58 | 58 | uv run ruff check --fix sdk/python/feast/ sdk/python/tests/ | |
| 59 | 59 | uv run ruff format sdk/python/feast/ sdk/python/tests/ | |
| 60 | 60 | ||
| 61 | + # File-aware format (for use with pre-commit, accepts file args) | ||
| 62 | + format-python-files: ## Format specified Python files | ||
| 63 | + @if [ -n "$(FILES)" ]; then \ | ||
| 64 | + uv run ruff check --fix $(FILES); \ | ||
| 65 | + uv run ruff format $(FILES); \ | ||
| 66 | + else \ | ||
| 67 | + echo "Usage: make format-python-files FILES='file1.py file2.py'"; \ | ||
| 68 | + fi | ||
| 69 | + | ||
| 61 | 70 | lint-python: ## Lint Python code | |
| 62 | 71 | uv run ruff check sdk/python/feast/ sdk/python/tests/ | |
| 63 | 72 | uv run ruff format --check sdk/python/feast/ sdk/python/tests/ | |
| 64 | 73 | uv run bash -c "cd sdk/python && mypy feast" | |
| 65 | 74 | ||
| 75 | + # File-aware lint (for use with pre-commit, accepts file args) | ||
| 76 | + lint-python-files: ## Lint specified Python files | ||
| 77 | + @if [ -n "$(FILES)" ]; then \ | ||
| 78 | + uv run ruff check $(FILES); \ | ||
| 79 | + uv run ruff format --check $(FILES); \ | ||
| 80 | + else \ | ||
| 81 | + echo "Usage: make lint-python-files FILES='file1.py file2.py'"; \ | ||
| 82 | + fi | ||
| 83 | + | ||
| 66 | 84 | # New combined target | |
| 67 | 85 | precommit-check: format-python lint-python ## Run all precommit checks | |
| 68 | 86 | @echo "✅ All precommit checks passed" | |
@@ -209,7 +227,7 @@ test-python-integration-local: ## Run Python integration tests (local dev mode) | |||
| 209 | 227 | HADOOP_HOME=$$HOME/hadoop \ | |
| 210 | 228 | CLASSPATH="$$( $$HADOOP_HOME/bin/hadoop classpath --glob ):$$CLASSPATH" \ | |
| 211 | 229 | HADOOP_USER_NAME=root \ | |
| 212 | - uv run python -m pytest --tb=short -v -n 8 --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadgroup \ | ||
| 230 | + uv run python -m pytest --tb=short -v -n auto --color=yes --integration --durations=10 --timeout=1200 --timeout_method=thread --dist loadscope \ | ||
| 213 | 231 | -k "not test_lambda_materialization and not test_snowflake_materialization" \ | |
| 214 | 232 | -m "not rbac_remote_integration_test" \ | |
| 215 | 233 | --log-cli-level=INFO -s \ | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -17,6 +17,9 @@ markers = | |||
| 17 | 17 | rbac_remote_integration_test: RBAC and remote functionality tests | |
| 18 | 18 | integration: Integration tests (slower, requires services) | |
| 19 | 19 | benchmark: Benchmark tests | |
| 20 | + slow: Tests taking >30 seconds | ||
| 21 | + cloud: Tests requiring cloud credentials | ||
| 22 | + local_only: Tests that run entirely locally | ||
| 20 | 23 | ||
| 21 | 24 | timeout = 300 | |
| 22 | 25 | timeout_method = thread | |
0 commit comments