| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Editing a cached function previously returned stale results because the cache key only contained the output name: redefining a function in a Jupyter notebook, or restarting after a code change with a persistent DiskCache, silently served results of the old implementation. - New `pipefunc.cache.compute_function_hash`: hashes the function's source (falling back to the code object for sourceless functions, recursing into `functools.partial` and callable instances' classes). - `PipeFunc._cache_id` now appends this fingerprint; `__pipefunc_hash__` still takes precedence when defined. If no fingerprint can be computed (e.g. builtins), a warning is emitted and the old behavior is kept. - `NestedPipeFunc` combines the fingerprints of its children. - Existing persistent caches are invalidated once on upgrade. Closes #964
✅ PR Title Formatted CorrectlyThe title of this PR has been updated to match the correct format. Thank you! |
Sorry, something went wrong.
Codecov Report✅ All modified and coverable lines are covered by tests.
... and 2 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Closes #964.
Problem
The cache key contained only the output_name and the input values — never the function's implementation (PipeFunc._cache_id only included a hash when the callable defined __pipefunc_hash__). Consequences:
This was the remaining source of the caching confusion reported in #902 (the mutation half was fixed in #905).
Solution
New pipefunc.cache.compute_function_hash(func) fingerprints a callable's implementation, and PipeFunc._cache_id appends it to the key. Resolution order:
functools.partial unwraps with its bound arguments folded in; callable class instances fingerprint their class (instance state is intentionally excluded — that is what __pipefunc_hash__ is for); NestedPipeFunc combines its children's fingerprints.
Design notes: source-hash chosen over bytecode-primary (human-predictable invalidation, stable across Python versions, works in notebooks; same approach as joblib.Memory) and over cloudpickle-bytes hashing (module functions pickle by reference → no invalidation; pickled set ordering interacts with hash randomization → nondeterministic keys across sessions).
Performance
Computed once per PipeFunc instance (_cache_id is a cached_property), lazily on first cache use: ~0.6 ms cold (file read), zero marginal cost per cache entry — per-entry key computation just reuses the precomputed string.
Breaking-ish
Existing persistent DiskCache entries are invalidated once on upgrade (keys changed). Arguably the point of the fix.
Limitation (documented)
Editing a helper function called by the cached function does not invalidate — same limitation as joblib; __pipefunc_hash__ is the escape hatch. Documented in the caching concept page.
Verification