| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
|
⚠️ Please install the Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## master #6592 +/- ##
==========================================
+ Coverage 46.35% 46.36% +0.01%
==========================================
Files 414 414
Lines 50052 50072 +20
Branches 7151 7154 +3
==========================================
+ Hits 23201 23218 +17
- Misses 25229 25231 +2
- Partials 1622 1623 +1
Continue to review full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
CliRunner.run() (used for 'feast apply' in unit tests) spawns a fresh child
interpreter that must import Feast (pandas/pyarrow/...) from cold. On loaded
CI runners — observed on unit-test-python (3.11, macos-14) — this can exceed
the 120s timeout, failing the test with:
AssertionError: Command timed out after 120s: ['apply']
Unlike the materializing paths guarded by run_with_output(), commands routed
through run() do not materialize and so cannot trigger the Dask atexit hang the
timeout was added for. A timeout here is therefore almost always transient cold
-start slowness, not a genuine deadlock, so retrying once (with warm OS/page
caches) clears it while a real repeated stall still fails after the attempts
are exhausted.
Builds on the CliRunner subprocess stabilization in feast-dev#6560.
Signed-off-by: Nikolaus Schuetz <nikolauspschuetz@gmail.com>
| Back | FazBrowse Home | New Git URL |
What changed
CliRunner.run() (the helper that runs feast apply in a fresh child interpreter for unit tests) now retries once on subprocess.TimeoutExpired instead of treating the first timeout as terminal. The per-attempt timeout is unchanged (120s); the timeout message now includes the attempt counter.
Why
The unit-test workflow flaked on unit-test-python (3.11, macos-14) while setting up test_feature_server.py::test_push_and_get:
run() spawns a new Python interpreter (sys.executable feast.cli apply) — deliberately, for feature-repo import isolation — which has to import Feast (pandas/pyarrow/protobuf/...) from cold. On a loaded macOS runner that cold start can occasionally exceed the 120s guard. The same commit passed on macos-14 / 3.12 and on all three Ubuntu legs, which is the signature of an environmental cold-start stall rather than a code fault.
Crucially, commands routed through run() (e.g. apply) do not materialize, so they cannot trigger the Dask atexit hang that run_with_output() and the module docstring describe. A timeout here is therefore almost always transient slowness, not a deadlock — so a single retry with warm OS/page caches clears it, while a genuine repeated stall still fails once the attempts are exhausted. Retries are intentionally not added to run_with_output(), whose commands can legitimately hang and whose existing machinery recovers partial output rather than retrying.
This builds directly on the CliRunner subprocess stabilization in #6560.
Validation
Note: the full unit suite could not be collected on my machine (local venv lacks the google.cloud test extra pulled by conftest.py); the change is confined to the retry loop, which is covered by the targeted verification above.
Separate from #6591 (which surfaced this flake).