FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

HydroModPy/tests/regression at refs/heads/dev · HydroModPy/HydroModPy · GitHub

/ HydroModPy Public

Breadcrumbs

  1. HydroModPy
  2. tests

regression

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History
 
 

Breadcrumbs

  1. HydroModPy
  2. tests

regression

README.md

Regression Tests (Golden References)

Scope and location

  • Canonical non-regression tests are under tests/regression/.

  • Golden references are under tests/regression/reference/golden_references/.

    • tests/regression/reference/golden_references/fast/
    • tests/regression/reference/golden_references/extensive/

  • Two tiers are used:

    • tests/regression/fast/ for routine checks
    • tests/regression/extensive/ for deeper end-to-end checks

  • Solver-family markers are used across launcher regression variants:

    • mf6 for MODFLOW 6 / GWT
    • nwt for MODFLOW-NWT / MODPATH / MT3DMS

  • Solver intercomparison tests use the intercomparison marker. They compare compact metrics produced by a [workflow].mode = "comparison" run, not figure files or full solver workspaces.

Current non-regression tests:

  • fast/test_simulation_regression_fast_boussinesq_regression.py
  • fast/test_simulation_regression_fast_boussinesq_divide_regression.py
  • fast/test_simulation_regression_fast_mf6_regression.py
  • fast/test_simulation_regression_fast_nwt_regression.py
  • fast/intercomparison/test_solver_intercomparison_fast_regression.py
  • extensive/test_simulation_regression_extensive_mf6_regression.py
  • extensive/test_simulation_regression_extensive_nwt_regression.py
  • extensive/test_launcher_data_overview_regression.py
  • extensive/test_run_geographic_case_regression.py

Notes:

  • Analytical and physical benchmark cases now live under tests/validation/ and validation_cases/.
  • Golden references are no longer meant to be shared between tiers when signatures differ.
  • Legacy folders test/ and test-old/ are no longer part of the active test workflow.

Run tests

Run all regression tests:

python -m pytest -m regression -q -n auto

Run only fast tier:

python -m pytest tests/regression/fast -q -n auto

Run only extensive tier:

python -m pytest tests/regression/extensive -q -n auto

Run only fast tests:

python -m pytest -m "regression and fast" -q -n auto

Run only extensive tests with marker:

python -m pytest -m "regression and extensive" -q -n auto

Run only slow tests:

python -m pytest -m slow -q -n auto

Run only NWT or MF6 launcher regressions:

python -m pytest -m "regression and nwt" -q -n auto
python -m pytest -m "regression and mf6" -q -n auto

Run only solver intercomparisons:

python -m pytest -m "regression and intercomparison" -q -n auto
hmp test regression --fast --intercomparison -j 1

Run one specific test:

python -m pytest tests/regression/fast/test_simulation_regression_fast_boussinesq_regression.py -q -n 1
python -m pytest tests/regression/fast/test_simulation_regression_fast_boussinesq_divide_regression.py -q -n 1
python -m pytest tests/regression/fast/test_simulation_regression_fast_mf6_regression.py -q -n 1
python -m pytest tests/regression/fast/test_simulation_regression_fast_nwt_regression.py -q -n 1
python -m pytest tests/regression/fast/intercomparison/test_solver_intercomparison_fast_regression.py -q -n 1
python -m pytest tests/regression/extensive/test_simulation_regression_extensive_mf6_regression.py -q -n 1
python -m pytest tests/regression/extensive/test_simulation_regression_extensive_nwt_regression.py -q -n 1

Run via the hmp CLI:

hmp test regression
hmp test regression --extensive
hmp test regression --fast
hmp test regression --slow
hmp test regression --list
hmp test regression simulation_regression_fast_mf6 --fast --mf6 -j 1
hmp test regression simulation_regression_fast_nwt --fast --nwt -j 1
hmp test regression simulation_regression_extensive_mf6 --extensive --mf6 -j 1
hmp test regression simulation_regression_extensive_nwt --extensive --nwt -j 1

Parallel execution (-n)

-n is provided by pytest-xdist and controls how many worker processes run tests in parallel.

  • -n auto: use all available CPU cores.
  • -n 4: force 4 workers.
  • -n 1: effectively serial execution with the xdist runner.

Examples:

python -m pytest -m regression -q -n auto
python -m pytest -m fast -q -n 4
python -m pytest tests/regression/extensive/test_simulation_regression_extensive_nwt_regression.py -q -n 1

Marker selection (-m)

-m accepts a boolean expression on test markers.

Operators:

  • and
  • or
  • not
  • parentheses (...)

Examples:

python -m pytest -m "regression and fast" -q
python -m pytest -m "regression and slow" -q
python -m pytest -m extensive -q
python -m pytest -m "regression and fast and mf6" -q
python -m pytest -m "regression and fast and intercomparison" -q
python -m pytest -m "regression and extensive and nwt" -q
python -m pytest -m "regression and not slow" -q
python -m pytest -m "(regression and fast) or slow" -q

Update golden references

Update all golden references:

python -m pytest -m regression -q -n auto --update-goldens

Update one golden file from one test:

python -m pytest tests/regression/fast/test_simulation_regression_fast_boussinesq_regression.py -q --update-goldens
python -m pytest tests/regression/fast/test_simulation_regression_fast_boussinesq_divide_regression.py -q --update-goldens
python -m pytest tests/regression/fast/test_simulation_regression_fast_mf6_regression.py -q --update-goldens
python -m pytest tests/regression/fast/test_simulation_regression_fast_nwt_regression.py -q --update-goldens
python -m pytest tests/regression/fast/intercomparison/test_solver_intercomparison_fast_regression.py -q --update-goldens
python -m pytest tests/regression/extensive/test_simulation_regression_extensive_mf6_regression.py -q --update-goldens
python -m pytest tests/regression/extensive/test_simulation_regression_extensive_nwt_regression.py -q --update-goldens

Notes

  • --update-goldens is a custom pytest option declared in tests/conftest.py.
  • Parallel execution requires pytest-xdist (-n auto).
  • MODPATH regression checks rely on .dbf snapshots, mainly the time column statistics.
  • CI should execute regression tests without --update-goldens.
  • Intercomparison tests should keep goldens compact: store selected metrics from comparison_metrics.json, while large figures remain documentation artifacts generated by the gallery/nightly documentation workflow.
Back | FazBrowse Home | New Git URL