fix(test): render the app test-runner HTML report (#3251) - #3258
Conversation
Sorry, something went wrong.
The built-in fallback app test runner (vendor/wheels/tests/app-runner.cfm, used when an app has no tests/runner.cfm of its own) emitted raw JSON for the no-format default and ?format=html, so opening /wheels/app/tests in a browser returned an unreadable JSON blob instead of the TestBox-style HTML report the core runner renders. This is item 1 of #3251 (items 2 and 3 shipped in #3255). - Extract the format-to-output decision into TestFormatResolver so the rule is unit-testable without HTTP (mirrors TestDirectoryResolver / TestDbResolver). The html / no-format branch now falls through to html.cfm with type="App" — a branch html.cfm already supported. json / txt / junit are unchanged, and an unrecognized ?format= value still produces no body, matching prior behavior. - Fix a latent Adobe ColdFusion 500 ("Routines cannot be declared more than once") in the shared report template html.cfm: its recursive processNestedSuites helper was a named function declaration, which leaks into the cached Public.cfc scope on Adobe and collides on the second include. It is now a variables-scoped function expression, the same pattern the core runner uses. This also fixes /wheels/core/tests?format=html on Adobe, which 500'd. - Add AppRunnerTestFormatSpec (resolver unit tests) and HtmlReportFunctionDeclarationGuardSpec (source guard against reintroducing a named function in html.cfm, since CI exercises the runners via format=json and never the html render path). Verified on Lucee 7 and Adobe 2023 (Docker): resolver + full dispatch area green (135 specs); /wheels/app/tests renders the HTML report and json/txt/junit are unchanged; format=html works on repeated requests on both engines. Fixes #3251 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Peter Amiri <petera@pai.com>
There was a problem hiding this comment.
Wheels Bot — Reviewer
TL;DR — This PR completes item 1 of #3251: /wheels/app/tests (and the no-format default) now renders the TestBox-style HTML report instead of raw JSON, and it fixes a latent Adobe ColdFusion Routines cannot be declared more than once 500 in the shared html.cfm template. The implementation is tightly scoped, mirrors the existing core-runner conventions exactly, ships a pure unit-testable resolver with TDD coverage plus a structural regression guard, and preserves every prior edge-case behavior. I traced each claim against the source and found no correctness, cross-engine, or security issues. Verdict: approve.
Correctness
Behavior preservation checks out across every branch:
- format=json|txt|junit are unchanged; the json branch keeps its 417-on-failure logic, Access-Control-Allow-Origin: * header, and injectScopeMetadata call (app-runner.cfm:139-161).
- Unrecognized/empty ?format= resolves to recognized=false, so testBox is never run and no body is emitted — identical to the old if/elseif chain that had no final else (app-runner.cfm:162-165, TestFormatResolver.cfc:79-86).
- The no-format→HTML switch does not break the CLI/MCP: every caller passes format=json explicitly (cli/lucli/Module.cfc:7854, cli/lucli/services/TestRunner.cfc:101, mcp-server.js.txt:400), so only browser users see the new HTML report — exactly the intent.
- The HTML branch correctly distinguishes a missing format key (→ HTML) from an empty format value (→ no output), and both are covered by specs.
Cross-engine
This change is a net positive here. The html.cfm helper conversion from a named declaration to variables.processNestedSuites = function(){...} (html.cfm:44-117) matches the prior-art pattern at vendor/wheels/tests/runner.cfm:20-23, whose comment explicitly cites avoiding "Adobe CF's DuplicateFunctionDefinitionException." Recursion via the variables-scoped reference (html.cfm:48) is safe on all engines. TestFormatResolver.cfc uses only switch/struct literals/LCase/Trim — no .map(), closures, or reserved-scope params. The guard spec's Left(trimmed, 2) / Left(trimmed, 1) use literal lengths (never 0), so Cross-Engine Invariant #8 does not fire.
Tests
- AppRunnerTestFormatSpec.cfc gives 10 BDD specs (extends wheels.WheelsTest) covering each format, case-insensitivity, trimming, and both unrecognized paths.
- HtmlReportFunctionDeclarationGuardSpec.cfc is a sound structural guard: I confirmed \bfunction\s+[a-zA-Z_] matches only the two //-prefixed comment lines in html.cfm (37-38), which the comment-skip logic drops, so the guard passes cleanly and won't false-positive on the inline-<script> IIFE or function expressions.
- Informational only (not blocking): the actual app-runner.cfm include wiring + statuscode is not exercised by an integration test, because CI hits the runners with format=json and never the html render path. The PR acknowledges this and mitigates it with the structural guard plus manual Lucee 7 / Adobe 2023 verification — a reasonable trade-off given the constraint.
Docs
Changelog fragment changelog.d/3251-app-test-html-report.fixed.md follows the <slug>.<type>.md convention (fixed) — no direct [Unreleased] edit. Inline comments are thorough and accurate.
Commits
fix(test): render the app test-runner HTML report (#3251) conforms to commitlint.config.js: valid type fix, valid scope test, subject ≤ 100 chars, not ALL-CAPS.
Security
No concerns. The Access-Control-Allow-Origin: * header is confined to the json branch exactly as before; the HTML branch is human-facing and adds no new header. No user input reaches SQL, and the directory scoping continues to flow through the existing TestDirectoryResolver allowlist.
Nice work — the latent-Adobe-500 catch and the accompanying guard are exactly the right instinct given that the format=html path is invisible to CI.
Sorry, something went wrong.
What & why
Resolves item 1 of #3251: /wheels/app/tests?format=html — and the no-format default — returned raw JSON instead of the TestBox-style HTML report the core runner (/wheels/core/tests) renders. A user opening the endpoint in a browser got an unreadable JSON blob.
Items 2 (subpath include) and 3 (testing-guide reconciliation) already shipped in #3255, so this PR completes #3251.
Design choice: render (not document)
The issue offered "render a real HTML report or document the JSON-only limitation." I chose render because the infrastructure was already built for it — html.cfm has had a dedicated type="App" branch (package=tests.specs, route=testbox) all along, and the testbox route resolves to /wheels/app/tests. The app runner simply never wired the fall-through. This is the completion of pre-existing work, and matches what the core runner does (type="Core").
Changes
TDD
Red → green on the resolver (watched the missing-CFC error, then the recognized-field refinement fail, then pass). The Adobe 500 was found via systematic debugging — captured the real exception (Routines cannot be declared more than once) and confirmed the original code also 500'd the core HTML report on Adobe, proving it pre-existing rather than introduced.
Verification (Lucee 7 + Adobe 2023, Docker)
Fixes #3251
🤖 Generated with Claude Code