PipeFunc
PipeFunc: Automate Python DAGs for Scientific Workflows - From Local to HPC with N-D Parameter Sweeps
pipefunc is a Python library that simplifies function composition and pipeline creation for scientific computing and data processing. It automatically generates DAGs from Python functions, efficiently handles N-dimensional parameter sweeps, and scales seamlessly from local to HPC environments. Key features include auto-parallization, N-dimensional map-reduce, and type checking. pipefunc minimizes boilerplate code, allowing users to focus on logic while it manages function dependencies and execution order. Ideal for complex workflows in data science, ML, and scientific computing.
Pinned Loading
Repositories
- pipefunc Public
Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪
- graphviz-anywidget Public
Interactive Graphviz visualization widget for Jupyter notebooks using anywidget using Graphviz as Wasm, d3-graphviz, and graphvizsvg
- xyflow-anywidget Public
- .github Public
People
This organization has no public members. You must be a member to see who’s a part of this organization.
Top languages
Loading…
Most used topics
Loading…