FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

ENH: Add `reuse_validation` parameter to `pipeline.map()` by basnijholt · Pull Request #920 · pipefunc/pipefunc · GitHub

ENH: Add reuse_validation parameter to pipeline.map() - #920

Merged
basnijholt merged 1 commit into
mainfrom
cleanup-validation
Nov 4, 2025
Merged

ENH: Add reuse_validation parameter to pipeline.map()#920
basnijholt merged 1 commit into
mainfrom
cleanup-validation

Conversation

Copy link
Copy Markdown
Collaborator

Add a new reuse_validation parameter to control validation strictness when reusing data from a previous run (cleanup=False).

This addresses the issue where users have input objects with broken __eq__ implementations that return incorrect results, preventing them from using the data reuse functionality.

Three validation modes:

  • "auto" (default): Warn if equality comparison fails, proceed anyway (maintains backward compatibility)
  • "strict": Raise error if equality comparison fails
  • "skip": Skip input/default validation entirely (for broken eq)

Note: Shapes and MapSpecs are always validated regardless of mode.

Changes:

  • Added reuse_validation parameter to Pipeline.map() and map_async()
  • Updated all internal run functions (run_map, run_map_eager, etc.)
  • Implemented validation logic in _compare_to_previous_run_info()
  • Added comprehensive test suite (8 tests, all passing)
  • Updated documentation for all affected functions

Add a new `reuse_validation` parameter to control validation strictness
when reusing data from a previous run (`cleanup=False`).

This addresses the issue where users have input objects with broken
`__eq__` implementations that return incorrect results, preventing them
from using the data reuse functionality.

Three validation modes:
- "auto" (default): Warn if equality comparison fails, proceed anyway
  (maintains backward compatibility)
- "strict": Raise error if equality comparison fails
- "skip": Skip input/default validation entirely (for broken __eq__)

Note: Shapes and MapSpecs are always validated regardless of mode.

Changes:
- Added `reuse_validation` parameter to Pipeline.map() and map_async()
- Updated all internal run functions (run_map, run_map_eager, etc.)
- Implemented validation logic in _compare_to_previous_run_info()
- Added comprehensive test suite (8 tests, all passing)
- Updated documentation for all affected functions

github-actions Bot commented Nov 4, 2025

Copy link
Copy Markdown
Contributor

✅ PR Title Formatted Correctly

The title of this PR has been updated to match the correct format. Thank you!

codecov Bot commented Nov 4, 2025
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
pipefunc/_pipeline/_base.py 100.00% <ø> (ø)
pipefunc/map/_prepare.py 100.00% <ø> (ø)
pipefunc/map/_run.py 100.00% <ø> (ø)
pipefunc/map/_run_eager.py 100.00% <ø> (ø)
pipefunc/map/_run_eager_async.py 100.00% <ø> (ø)
pipefunc/map/_run_info.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codspeed-hq Bot commented Nov 4, 2025

Copy link
Copy Markdown

CodSpeed Performance Report

Merging #920 will not alter performance

Comparing cleanup-validation (d89c861) with main (75db630)

Summary

✅ 6 untouched

basnijholt merged commit cd662ab into main Nov 4, 2025
21 checks passed
basnijholt deleted the cleanup-validation branch November 4, 2025 22:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL