This increment demonstrates a small, reproducible Data/BI preparation workflow rather than a production ETL platform. It turns a deliberately imperfect synthetic CSV into four auditable outputs:

1. cleaned assessment results
2. rejected rows with explicit reason codes
3. module-level KPIs
4. a machine-readable quality report

The implementation is intentionally separated into reusable functions so validation and transformation logic can be tested independently from file-system export.

The raw CSV must contain these columns:

Unexpected columns are ignored but listed in quality_report.json.

The workflow performs the following steps in a fixed order:

1. read all CSV fields as text
2. verify required columns
3. preserve the original CSV row number as source_row
4. trim text, uppercase identifiers and collapse repeated module whitespace
5. convert date and numeric fields with invalid values becoming rejection reasons
6. enforce score and threshold ranges
7. remove later copies of exact duplicate result_id rows
8. reject every variant of a conflicting duplicate result_id
9. derive score_percentage and Boolean passed
10. aggregate deterministic module KPIs
11. export CSV and JSON outputs

Invalid rows are not silently discarded. They remain visible in rejected_results.csv with pipe-separated reason codes.

data/raw/training_results.csv intentionally contains:

• one exact duplicate
• a missing score
• a score above the maximum
• an invalid date
• a missing learner identifier
• a non-positive maximum score
• a pass threshold above the maximum
• whitespace requiring normalisation

The committed raw data is synthetic and contains no personal information.

From the repository root:

python -m data_quality \
  --input data/raw/training_results.csv \
  --output .ci-output/data-quality

PowerShell uses the same arguments:

python -m data_quality `
  --input "data/raw/training_results.csv" `
  --output ".ci-output/data-quality"

The .ci-output/ directory is already ignored by Git.

The committed fixture contains 15 input rows. The expected workflow result is 8 analysis-ready rows and 7 rejected rows, including the later copy of the exact duplicate.

Expected module KPI control values include:

A missing input file or missing required column stops the workflow with a non-zero exit code. Row-level quality problems do not crash the workflow; they are isolated in the rejection output and summarised in the report.

This is a learning-grade quality workflow for small CSV files. It does not claim streaming ingestion, distributed processing, schema evolution, database transactions, orchestration, production observability or regulatory validation.