| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Signed-off-by: HaoXuAI <sduxuhao@gmail.com>
|
⚠️ Please install the Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## master #6958 +/- ##
==========================================
+ Coverage 49.32% 49.69% +0.36%
==========================================
Files 443 443
Lines 55094 55094
Branches 8017 8017
==========================================
+ Hits 27176 27378 +202
+ Misses 26035 25814 -221
- Partials 1883 1902 +19
Continue to review full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
|
Need to run make lock-python-dependencies-all and pixi lock |
Sorry, something went wrong.
|
Thanks — you are right on both counts, and I want to be straight that I cannot run either command from my environment. Here is what I was able to establish, in case it saves you time. pixi lock is required, and I initially thought it would not be. I assumed a new PyPI extra could not affect it, since none of the three [tool.pixi.feature.*.pypi-dependencies] entries references ci or lance — they use aws/gcp/..., duckdb/delta/... and ray/..., so nothing in the resolved package set changes. That reasoning was incomplete: feast is an editable path dependency, so pixi.lock records its full requires_dist, including this line: - feast[aws,azure,cassandra,...,iceberg,image,k8s,mcp,milvus,...] ; extra == 'ci' which is the pre-change expansion and does not contain lance. So the lock does need regenerating, but the change should be confined to that one block: the ci list gains lance, plus two new ; extra == 'lance' entries for pylance and lance-namespace. No resolution change, because no feature pulls those extras. I deliberately did not hand-edit the file, since a hand edit that does not match the generator is worse than none. For the requirements, only py3.{10,11,12}-ci-requirements.txt are affected. lance is referenced solely by the ci extra, and those are the only requirement files containing torch: py3.10-ci-requirements.txt torch=2 py3.10-minimal-requirements.txt torch=0 py3.10-minimal-sdist-requirements.txt torch=0 py3.10-minimal-sdist-requirements-build.txt torch=0 py3.10-requirements.txt torch=0 I confirmed the non-ci files are unaffected by regenerating py3.10-requirements.txt on this branch: zero pylance/lance-namespace lines appear. One thing worth deciding before the refresh runs. I regenerated py3.10-requirements.txt from master's unmodified pyproject.toml as a control, and it already differs from the committed file by 1551 lines and 14 version pins — fastapi 0.141.1 → 0.142.2, mypy 2.3.1 → 2.4.0, plus markupsafe, mmh3, pyjwt, python-dotenv, pytz, charset-normalizer, librt, ast-serialize and a new opentelemetry-api transitive. That is upstream drift since the last refresh, not anything this PR introduces, but it means make lock-python-dependencies-all run today will bundle those bumps into this PR whoever runs it. Happy either way — I just did not want to hand you a dependency PR carrying a dozen unrelated upgrades without flagging it. Why I cannot run it. pixi run needs conda.anaconda.org and the --torch-backend cpu compile needs download.pytorch.org, which then fans out to download-r2.pytorch.org; all three are blocked by my network egress policy, and add_cpu_torch_hashes.py invokes --torch-backend cpu a second time, so there is no way around the PyTorch index. Everything else resolves fine, so this is specifically the conda and PyTorch hosts. Could you run make lock-python-dependencies-all and pixi lock on this branch, or would you prefer I close this and have someone fold the extra into a refresh you were going to do anyway? I am glad to do any other part of it. |
Sorry, something went wrong.
|
Follow-up: I tried running both commands and make lock-python-dependencies-all cannot complete on master, independently of this PR. ValueError: Missing CPU hashes for torch==2.14.1+cpu make: *** [lock-python-dependencies-all] Error 1 add_cpu_torch_hashes.py looks each torch entry up by appending +cpu to its version, but a --universal --torch-backend cpu resolve emits two pins per package, and the second is already +cpu: torch==2.14.0 ; sys_platform == 'darwin' torch==2.14.0+cpu ; sys_platform != 'darwin' so it asks for 2.14.0+cpu+cpu and raises. That shape is in all three committed py3.*-ci-requirements.txt today, so it is not something this branch introduced. The script and its tests landed together in #6188 and the requirements were last regenerated before that, and the existing test feeds in a bare torch==2.13.0, which is why it went unnoticed. Fix is up as #6959, with regression tests for the split shape and a check against all three committed files. Once that merges I will rerun the lock target here and push the result. One note for whoever regenerates: the target starts with rm -rf sdk/python/requirements/*, so a failed run leaves 14 of the 15 files deleted. Worth a git checkout -- sdk/python/requirements/ afterwards. pixi lock itself works and is unaffected. |
Sorry, something went wrong.
`feast` is an editable path dependency, so `pixi.lock` records its full `requires_dist`, including the expansion of every extra. Adding the `lance` extra therefore changes the lock even though no pixi feature consumes it. The change is confined to that one block: the `ci` expansion gains `lance`, and the two `lance` requirements are recorded. No package resolves differently, which is what the diff size shows -- 3 insertions and 1 deletion, and the 1377 pypi and 211 conda artifact URLs are unchanged. Signed-off-by: hao-xu5 <hxu44@apple.com>
|
pixi lock is done and pushed as df2b0ec. The change is confined to the one block I expected, because feast is an editable path dependency and so the lock records its full requires_dist including every extra's expansion: + - pylance>=12.0.0 ; extra == 'lance'
+ - lance-namespace>=0.11.1 ; extra == 'lance'
- - feast[...,k8s,mcp,...] ; extra == 'ci'
+ - feast[...,k8s,lance,mcp,...] ; extra == 'ci'3 insertions, 1 deletion. Nothing resolves differently — the 1377 pypi and 211 conda artifact URLs are byte-identical to before, which is expected since none of the three pixi features consumes the ci or lance extra. make lock-python-dependencies-all is still blocked on #6959. I will rerun it and push the requirements here once that merges. |
Sorry, something went wrong.
Signed-off-by: HaoXuAI <sduxuhao@gmail.com>
|
Updated in afe2961:
I also retried make lock-python-dependencies-all. With the #6959 fix applied locally it gets past the CPU torch hash issue, but the target then fails in the unrelated minimal-sdist build step because pybuild-deps==0.5.0 calls pip-tools with an incompatible OutputWriter(generate_hashes=...) API. Since lance is only included through the ci extra, the 12 non-CI requirement files are unaffected; I restored those rather than committing unrelated resolver drift. Local verification after the final master sync:
|
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Summary
Adds an installable lance extra for the native Lance read/write paths:
This is split from #6943 so the data-source implementation and packaging change can be reviewed independently.
Verification
Related: #6943, #6945