A self-contained Retrieval-Augmented Generation pipeline over Verordnung (EU) 2024/1689 (KI-Verordnung), built with open-source models and FAISS.

View the accompanying tutorial talk (in German): https://sumeromer.github.io/rag-tutorial/

conda env create -f environment.yaml
conda activate rag-tutorial

Or, with an existing Python 3.12 environment (the dependencies are declared in pyproject.toml, which environment.yaml installs from):

pip install -e ".[dev]"

python run.py build

On first run the EU AI Act HTML (~1.3 MB) is downloaded automatically from EUR-Lex and cached to data/eu_ai_act_de.html. The FAISS index is then built over ~460 chunks with BAAI/bge-m3 and saved to faiss_index/.

Expected build time: ~5–15 min on CPU · ~2–5 min on GPU/MPS.

# with RAG (default)
python run.py ask "Welche KI-Praktiken sind nach Artikel 5 verboten?"

# without RAG — model knowledge only
python run.py ask "Welche KI-Praktiken sind nach Artikel 5 verboten?" --no-rag

# side-by-side comparison
python run.py ask "Welche KI-Praktiken sind nach Artikel 5 verboten?" --compare

Note: This is not a rigorous evaluation; these 15 questions serve qualitative illustration and tutorial purposes only. A quantitative assessment of how much retrieval improves answer accuracy and reliability would require a larger, representative benchmark spanning the breadth of the EU AI Act.

python compare.py

Runs 15 sample questions (10 in-scope + 5 out-of-scope) with and without RAG. Results are saved incrementally to data/comparison_output.json after each question, so a partial run is not lost.

All models are downloaded automatically on first use via Hugging Face.

Tested on Apple M2 Max (96 GB) with MPS backend (PYTORCH_ENABLE_MPS_FALLBACK=1).
On CUDA: bitsandbytes is installed for optional quantization (not enabled by default).
On CPU only: generation is very slow (~5–10 min/question).

.
├── src/
│   ├── config.py      # paths, model names, device selection
│   ├── load.py        # HTML download + text normalization
│   ├── chunk.py       # structural + Absatz-level chunking
│   ├── retrieve.py    # FAISS index, BM25, RRF fusion, reranker
│   └── generate.py    # Qwen2.5-14B reader + system prompt
├── data/
│   └── eu_ai_act_de.html        # auto-downloaded on first run
├── faiss_index/                 # created by `python run.py build`
├── slides/
│   ├── 2026-05-29-RAG-Tutorial.qmd   # public deck (speaker notes stripped)
│   └── build-public.py              # regenerates the public deck from the local one
├── run.py                       # CLI: build / ask / compare
├── compare.py                   # 15-question benchmark
├── pyproject.toml               # dependencies + ruff config
└── environment.yaml             # conda env (installs from pyproject.toml)

Dependencies live in pyproject.toml (single source of truth). Lint with ruff:

ruff check .          # report issues
ruff check . --fix    # apply safe auto-fixes

The slide deck has two versions: a private *.local.qmd (git-ignored) that contains the speaker notes, and the committed 2026-05-29-RAG-Tutorial.qmd with those notes stripped. Regenerate the public deck after editing the local one:

python slides/build-public.py

[
  {
    "id": 1,
    "scope": "in-scope",
    "question": "Was ist das Hauptziel der KI-Verordnung der EU?",
    "with_rag": "...",
    "sources": ["Erwägungsgrund (1)", "Artikel 1 — Gegenstand"],
    "without_rag": "..."
  },
  ...
]