This toolchain generates the static illustrated capability gallery used by the
Sphinx documentation.
Edit versioned inputs here:
- tools/doc_gallery/manifests/
- examples/projects/09_capability_gallery/
- examples/projects/07_mesh_gallery/
- the source configs and launchers referenced by each case
Do not hand-edit generated docs artifacts here:
- docs/source/capability_gallery/
- docs/source/_static/capability_gallery/
Those files are regenerated by python -m tools.doc_gallery.
Generate or refresh the committed gallery artifacts:
python -m tools.doc_gallery
For the whole Read the Docs refresh chain, use the dedicated orchestration
script. It recomputes validation reports, the XT3D note payload, the gallery
artifacts, runs the gallery drift check, then rebuilds Sphinx locally:
python -m tools.refresh_readthedocs
Add --install-solver-binaries on clean CI machines that do not already have
the MODFLOW executables cached:
python -m tools.refresh_readthedocs --install-solver-binaries
To enforce that such a refresh only touched the generated docs artifacts, use:
python -m tools.verify_docs_refresh_outputs
The scheduled GitHub workflow .github/workflows/docs-nightly-refresh.yml
chains both commands before committing the refreshed artifacts back to the
documentation branches.
On Windows, one wrapper can chain the common local workflow:
- optional analytical validation report refresh
- capability-gallery regeneration
- gallery drift check
- Sphinx HTML rebuild in docs/build/html by default
powershell -File tools/update_capability_gallery.ps1
When one step needs the scientific stack, the wrapper auto-prefers
conda run -n hydromodpy-kpg python if that environment exists. Override it
explicitly when needed:
powershell -File tools/update_capability_gallery.ps1 `
-CondaEnv hydromodpy-kpg
For one rerun-focused refresh, keep the scope narrow:
powershell -File tools/update_capability_gallery.ps1 `
-Only modflow6_irregular_tri_xt3d_method_choice `
-OpenHtml
The XT3D irregular-triangle diagnostics note uses its own committed report
source. Targeting that slug now refreshes the report automatically before the
gallery rebuild, or you can force that step explicitly:
powershell -File tools/update_capability_gallery.ps1 `
-IncludeXt3dDiagnostics `
-Only modflow6_irregular_tri_xt3d_method_choice
Use -BuildDir _build/html only if you explicitly want the alternate Sphinx
output tree; build/html matches make.bat and the usual local browser path.
Verify that the committed generated files are in sync with the manifest and the
source hashes:
python -m tools.doc_gallery --check
List the currently declared cases, or only one category / one slug:
python -m tools.doc_gallery --list
python -m tools.doc_gallery --list --category geographic
python -m tools.doc_gallery --list --only geographic_nancon_identity_card
Refresh only one case or one category while keeping the rest of the committed
gallery as baseline:
python -m tools.doc_gallery --only geographic_nancon_identity_card
python -m tools.doc_gallery --category geographic
python -m tools.doc_gallery --check --only geographic_nancon_identity_card
The gallery now uses one repo-local temporary root by default:
- .tmp-doc-gallery/ under the repository root
- override with HMP_DOC_GALLERY_TMPDIR if needed
Scaffold one new declarative copy_assets case:
python -m tools.doc_gallery.new_case \
--manifest geographic_cases.json \
--category geographic \
--slug geographic_example_case \
--title "Example Geographic Case"
Import one local mesh bundle into the canonical repository layout used by the
gallery:
python -m tools.doc_gallery.import_mesh_bundle \
--source-bundle C:/results/HydromodPy/mesh_catchment_runs/headwater_100km2/mesh_outlet_27/mesh_catchment_outlet_27_bundle \
--scale 100km2 \
--variant geology_rivers_buffer30 \
--outlet-id 27
Sync the repeated mesh-gallery families directly from existing batch results and
refresh the generated docs in one go:
python -m tools.doc_gallery.sync_mesh_catchment_runs --update-gallery
The generator rewrites:
- docs/source/capability_gallery/
- docs/source/_static/capability_gallery/
The documentation build does not execute the gallery cases. It only reads the
committed .rst, .png, and .json artifacts generated ahead of time.
For PETSc-backed Boussinesq figures, keep the same rule locally. Produce the
numerical artifacts in WSL, then rebuild the documentation in Windows:
wsl.exe bash -lc "cd /mnt/c/codes/HydroModPy && bash install/enter_wsl_dev.sh --headless -- bash tools/ci/run_boussinesq_petsc_smoke.sh"
wsl.exe bash -lc "cd /mnt/c/codes/HydroModPy && bash install/enter_wsl_dev.sh --headless -- python -m tools.doc_gallery.generate_boussinesq_drying_assets"
conda run --no-capture-output -n hydromodpy-kpg python -m sphinx -E -a -W -b html docs/source docs/build/html
The Sphinx step should not import petsc4py or rerun PETSc simulations.
Case inventory now lives in two places:
- tools/doc_gallery/gallery_manifest.py for generator-backed cases whose
metadata still benefits from Python helpers or discovery code,
- tools/doc_gallery/manifests/*.json for small declarative inventories,
especially stable copy_assets cases where adding one page should mostly be
data entry rather than Python editing.
Each GalleryCaseSpec declares:
- the category and page metadata
- optional guided-doc links for onboarding
- optional key-parameter and reading-order notes
- the reproduction command shown in the docs
- the source files tracked for staleness detection
- the generator kind (mesh_viewer, copy_assets, or validation_case)
- the image assets and displayed metrics
For future mesh-gallery cases, the canonical repository input tree lives under
examples/projects/07_mesh_gallery/.
- tools/doc_gallery/import_mesh_bundle.py copies one local bundle into that tree
- tools/doc_gallery/sync_mesh_catchment_runs.py bulk-refreshes repeated mesh families from C:/results/Hydromodpy/mesh_catchment_runs/
- tools/doc_gallery/mesh_case_registry.py defines the shared case schema and naming
- tools/doc_gallery/gallery_manifest.py auto-discovers examples/projects/07_mesh_gallery/**/case.json
Simple committed asset-copy cases can also be declared through JSON manifests
under tools/doc_gallery/manifests/.
- the current code_comparison pages use this path,
- the current geographic copy-assets pages use it too,
- JSON manifests are a good fit when the generator is already known and the work
is mostly title/summary/assets/metadata declaration,
- Python stays the right place for cases that need helper builders, discovery,
metric formatter functions, or richer derived defaults.
- tools/doc_gallery/manifests/README.md documents the expected JSON fields and
the guardrails enforced by the loader.
Published simulation-comparison cases can be staged under
examples/projects/09_capability_gallery/simulation_comparison/.
- tools/doc_gallery/import_simulation_comparison.py copies a reviewed
comparison_manifest.json and comparison_metrics.json bundle into that
versioned tree as comparison_manifest.json plus compact summary_metrics.csv.
Published text artifacts are normalized to the current simulation_regression
path vocabulary during import.
It can publish one comparison root or
discover every comparison under a testbed output root, including the Nançon
hydrographic-network benchmark's comparison/ directory.
- tools/doc_gallery/gallery_simulation_comparison_specs.py discovers every
published <slug>/case.json and regenerates the doc page from committed
artifacts only.
- generated *_observables.csv files under
docs/source/_static/capability_gallery/simulation_comparison/ are public
snapshots and intentionally omit workspace-local source_path and
run_folder columns.
- generated *_comparison_manifest.json files are also publication-safe:
gallery generation and bundle import remove workspace-local path fields,
generated config pointers, and captured stdout/stderr tails before writing
them under docs/source/_static/capability_gallery/simulation_comparison/.
- this keeps Sphinx and python -m tools.doc_gallery --check free of heavy
solver execution; stale bundles should be refreshed by an explicit comparison
regeneration step before the docs are rebuilt.
Analytical validation cases are discovered automatically from
validation_cases/analytical/.
- tools/doc_gallery/validation_case_registry.py reads metadata.toml, the
case README.md, and the global inventory in validation_cases/README.md
- solver coverage is inferred from [config_files]
- gallery pages render one common benchmark description plus solver-specific
tabs when a case exposes more than one backend
- For a simple copy_assets page, prefer adding one entry under tools/doc_gallery/manifests/*.json.
- For a generated or discovered case, add the corresponding GalleryCaseSpec or helper builder in tools/doc_gallery/gallery_manifest.py.
- Make sure the case is reproducible from versioned repository inputs rather than one local results_stable tree.
- If it is a new declarative copy_assets case, start with python -m tools.doc_gallery.new_case ... to create the manifest entry and asset directory.
- Run python -m tools.doc_gallery --list --only <slug> to confirm the case is visible in the inventory.
- Run python -m tools.doc_gallery --only <slug>.
- Inspect the generated page under docs/source/capability_gallery/cases/.
- Run python -m tools.doc_gallery --check --only <slug>.
- Rebuild Sphinx with python -m sphinx -E -a -W -b html source _build/html from docs/.
How To Add One Mesh Bundle Case
- Produce one local bundle with the mesh launcher.
- Import it into examples/projects/07_mesh_gallery/ with python -m tools.doc_gallery.import_mesh_bundle ....
- Review the generated case.json, viewer_config.toml, and README.md.
- Run python -m tools.doc_gallery.
- Rebuild Sphinx and inspect the new page under capability_gallery/mesh.
How To Add One Validation Case
- Add the case under validation_cases/analytical/ with README.md,
metadata.toml, comparison.py, plotting.py, and run_case.py.
- Register the case in the inventory tables of validation_cases/README.md.
- If the benchmark belongs to a new analytical family, add its equations in
tools/doc_gallery/validation_case_registry.py.
- Run python -m tools.doc_gallery.
- Rebuild Sphinx and inspect the page under capability_gallery/validation.
The repository now carries a dedicated GitHub Actions workflow:
- .github/workflows/docs-gallery-check.yml
Its job is intentionally narrow:
- install the lightweight Python environment required by the gallery tooling,
- run python -m tools.doc_gallery --check --category mesh,geographic,simulation_comparison,code_comparison,
- fail the PR if committed gallery artifacts drift away from the declarative
inventory or tracked source hashes.
It does not run any solver: the checked categories are rebuilt from versioned
repository inputs only. The validation and calibration categories stay out of
that job because they need solver binaries and heavier scientific runtimes.