MRX-72 — adversarial testing, low-level optimization, and quantitative research.

Work merged into upstream security tooling.

Setup. Four production models were each run through the same 107 attack vectors covering all ten OWASP LLM Top 10 categories. Same system prompt, temperature 0, zero errors across every run. Detection was deterministic: a random secret ("canary") was planted in the system prompt, and a vector counted as a bypass only if the model's output contained it (or a fixed leak pattern). No LLM judge.

  Vectors bypassed, out of 107

  gemini-3.1-flash-lite  ███████████████  15/107
  gpt-oss-20b            ██████████·····  10/107
  gpt-oss-120b           █████··········  5/107
  qwen3.8-27b            █████··········  5/107

Where the 35 failures landed:

  agentic & output layer   ████████████████████████████████████  34
  everything else          █                                      1

  jailbreaks · encoding bypass · indirect injection
  · system-prompt extraction  (51 vectors)              zero bypasses

Finding. The attacks everyone writes about failed completely. 51 vectors covering jailbreaks, encoding bypass (base64, ROT13, homoglyphs), indirect injection and system-prompt extraction got 0 bypasses on all four models.

34 of the 35 failures landed in the agentic and output layer instead:

• Tool misuse: calling transfer_funds with no confirmation, running a send_email call that was hidden inside a pasted ticket, using an admin token when a normal tool would have worked
• Output handling: filling a secret into a markdown image URL, which leaks it with zero clicks as soon as the chat renders
• PII disclosure: asked to reformat a config file as JSON, all four models returned the API keys unredacted
• Confabulation: inventing a pip package name, which opens the door to slopsquatting
• Unbounded consumption: getting pushed into runaway repetitive output

Takeaway: refusal training works on the prompts it was trained on. The risk sits after the model decides to help: in what it writes, which tools it calls, and what data it repeats back. The same model that refuses a poisoned document telling it to say something will obey one telling it to do something.

One run per model at temperature 0, so these are observations and not rates · synthetic system prompt · the suite has since grown to 330 vectors, and the new ones have not been run yet · every raw report and per-finding transcript is public

Full evaluation →  ·  Raw data →

Built and maintained solo. All open source.

Red-teams an LLM app against the OWASP LLM Top 10 and reports exactly which attacks got through. A random canary planted in the system prompt makes every finding a reproducible string match: one API call per vector, no second LLM to judge. Works with any provider via LiteLLM.

lrtf scan gpt-4o --tui               # live view as each vector lands
lrtf compare gpt-4o claude-sonnet-4-5 ollama/llama3
lrtf diff base.json current.json     # did your fix actually work?

Backtests strategies and builds portfolios, with the statistical checks that flag an overfit result. Signals are lagged one bar (no lookahead), costs are charged on turnover, and every result is benchmarked to buy-and-hold. Walk-forward validation, Black-Litterman allocation, and bootstrap significance tests on the Sharpe.

qfcli --backtest AAPL --walk-forward --ensemble rank --grid "fast=10,20;slow=40,60"
qfcli --portfolio AAPL MSFT NVDA --bl --view NVDA=0.18 --ff

A dependency-free TCP port scanner in C++17 on raw BSD sockets. It never shells out to nmap. Non-blocking connect() with poll() timeouts, a bounded worker pool, banner grabbing, and stable text/JSON/CSV output. CI runs ASan/UBSan on macOS and Linux.

zapscan -p 1-1024 -c 256 scanme.nmap.org
zapscan -j -o report.json -p 22,80,443 10.0.0.0/24

LLM security   Prompt injection, jailbreaks, encoding bypass, RAG injection and multi-turn chains, scored by deterministic canary detection.

Agentic AI security   Memory poisoning and excessive-agency abuse across LangChain, ChromaDB and Mem0, and the integrity checks that catch it.

Systems & offensive security   POSIX-socket network tooling, x86_64 assembly, email forensics, and dependency CVE analysis via OSV.

Quantitative finance   Backtesting that catches its own overfitting: no lookahead, costs on turnover, walk-forward validation.

• Shipped — LRTF: an LLM red-team CLI. 330 vectors mapped to the OWASP LLM Top 10, deterministic canary detection, multi-turn attack chains, multi-provider through LiteLLM.
• Building — AMPAF, an agentic memory-poisoning framework. Four payload classes (identity shift, behavior drift, data exfiltration, bias injection) against LangChain, ChromaDB, and Mem0, with an integrity checker, anomaly detector, and live memory-state monitor.
• Direction — a full-lifecycle AI security toolchain: pre-deployment testing, runtime defense, incident forensics.
• Collaborator — OWASP.
• Core dev — Omnikon.

theaiguy369@gmail.com