TerminalBench Bot Commands

Run tasks:
/bot tb run [--dataset, --dataset-path, --dataset-config, --registry-url, --local-registry-path, --output-path, --run-id, --upload-results, --task-id, --n-tasks, --exclude-task-id, --no-rebuild, --cleanup, --use-subscription, --model, --agent, --agent-import-path, --agent-kwarg, --log-level, --livestream, --n-concurrent, --n-attempts, --global-timeout-multiplier, --global-agent-timeout-sec, --global-test-timeout-sec, --contributionsCommit]

Check:
/bot tb tasks check [--task-id, --tasks-dir, --unit-test-relative-path, --dockerfile-relative-path, --model, --agent, --fix, --output-path, --contributionsCommit]

Debug:
/bot tb tasks debug [--task-id, --run-id, --runs-dir, --tb-run-job-id, --tasks-dir, --agent, --model, --n-trials, --output-path, --contributionsCommit]

Full Check:
/bot full-check-v2 [--task-id <id>] [--analyze-failure] [...]
/bot full-check-v2 --opus-only - Run only tb_run_large with Claude Opus 4.7
/bot full-check-v2 --sonnet-only - Run only tb_run_large with Claude Sonnet 4.5
/bot full-check-v2 --tbench - Oracle + NOP + tb_run_large (Codex + openai/gpt-5.5, 5 parallel attempts, subscription; requires 0-3/5 resolved) + Codex harbor debug/analyze when ≤3/5 resolved
/bot full-check-v2 --tbench --opus - Same as --tbench, but tb_run_large uses Claude Opus 4.8 via claude-code
/bot full-check-v2 --fusion-reports - Oracle + NOP + harbor_run_large (Codex + openai/gpt-5.5, 1 attempt, subscription). No fallback stage. Harbor saves /output, /app/output, and traces per trial automatically (implicit --artifacts — needed for downstream re-verify).
/bot full-check-v2 --oracle-nop-only - Lightweight gate: only Oracle (5 attempts, 5 retries) + NOP, skip every model run / similarity / debug / quality check (~30-60s per task)
/bot full-check --tb-run-large-agent claude-code --tb-run-large-model claude-opus-4-7

Grok Trace Run:
/bot run-grok --5 - Shortcut for Oracle + NOP + 5 Grok Build trials with artifacts/traces.
/bot run-grok --5 --appends - Add 5 Grok traces after the existing S3 traces, then auto-rescore the full S3 trace set.
/bot run-grok --8 - Shortcut for Oracle + NOP + 8 Grok Build trials with artifacts/traces.

Trace Run:
/bot mm-trace-run - Oracle + NOP + 5 Codex GPT-5.5 xhigh trials (subscription) with artifacts/traces. Posts per-trial rewards and step counts, avg reward, avg steps, max/avg reward ratio, and artifact + trace viewer links.

Re-verify (re-score existing agent attempts against updated tests):
/bot re-verify [--task-id <id>] [--skip-oracle] [--skip-nop] - Re-run tasks/<task-id>/tests/test.sh against every trajectories*/<task-id>/<agent>/<N>/artifacts/output/ directory in the PR head ref (1 claude + 4 grok by convention). Also runs oracle (canonical solution/solve.sh, expected reward 1.0) and nop (empty /output, expected reward 0.0) sanity rows by default — pass --skip-oracle or --skip-nop to opt out. Produces fresh per-attempt rewards without re-running the agents — useful after editing tests/ during review.
/bot rescore [--trace-s3-url s3://bucket/prefix] - Re-score existing S3 trace outputs against current tests after rubric/test-only changes; runs oracle + nop sanity rows by default and does not use PR-committed trajectories.
/bot fairness-review - Run the structured task fairness review and render a standardized PASS/WARN/FAIL comment.

Harbor format checker:
/bot harbor-format-check [--trace-s3-url s3://bucket/prefix[,s3://bucket/other-prefix]] [--policy mm-abc|compat] - Run the standardized pre-acceptance format checker on this PR, including LLM fuzzy checks and optional S3 trace checks.

Offline-search reviewer:
/bot offline-search-review [--agents 1-5] - Run the offline-search audit reviewer on this PR and post the auditrobot summary back here.
/bot offline-review [--agents 1-5] - Short alias for /bot offline-search-review.

Sapphire format checker:
/bot sapphire-format-check [--task-dir tasks/<id>] [--no-llm] [--check-traces] - Run the mm-sapphire-pipelines format checker on this PR without touching full-check or mm-trace-run. Trace checks are opt-in.

For detailed parameter descriptions, run tb --help or tb <command> --help locally.

Job Management:
/bot job list - List all running jobs
/bot job status <job_id> - Get status of a specific job
/bot job kill <job_id> - Kill a running job
/bot job restart <job_id> - Restart a failed job
/bot job info <job_id> - Show detailed information about a job
/bot job cleanup - Remove all failed-to-report jobs

Review:
/bot code-review - Trigger the generic AI code review service on this PR

Remove default flags: Use --no-{flag} to disable default flags (e.g., --no-use-subscription)

Aliases:
/bot /codex-attempts [--n, --task-id, --tasks-dir] → /bot /tb run --agent codex
/bot /claude-attempts [--n, --task-id, --tasks-dir] → /bot /tb run --agent claude-code
/bot /grok-attempts [--n, --task-id, --tasks-dir] → /bot /tb run --agent terminus-2 --model xai/grok-4.3-internal --agent-kwarg reasoning_effort=high
/bot /tb-check [--task-id, --tasks-dir] → /bot /tb tasks check
/bot /oracle [--task-id, --tasks-dir] → /bot /tb run --agent oracle
/bot /nop [--task-id, --tasks-dir] → /bot /tb run --agent nop
/bot /tb-debug [--task-id, --tasks-dir] → /bot /tb tasks debug

Get help: /help or /bot help