test_runner: add statement coverage support by Felipeness · Pull Request #62340 · nodejs/node · GitHub

Summary

Add statement coverage to the test runner's coverage reporting, alongside existing line, branch, and function coverage
Parse source files with acorn to extract AST statement nodes, then map V8 coverage ranges to those statements using a most-specific-range algorithm
Add --test-coverage-statements CLI option for setting minimum statement coverage thresholds
Files that cannot be parsed by acorn gracefully degrade to 100% statement coverage

Implementation

Uses acorn-walk's Statement visitor to collect all non-BlockStatement nodes from the AST. For each statement, the algorithm finds the most specific (smallest) V8 coverage range that fully contains it, using that range's execution count as the statement's coverage count. This approach is similar to how @aspect-build/v8-coverage (used by Vitest) handles statement coverage.

Files changed

lib/internal/test_runner/coverage.js - Core statement extraction and coverage computation
lib/internal/test_runner/utils.js - Added stmts % column to coverage report table
lib/internal/test_runner/test.js - Statement coverage threshold checking
src/node_options.cc / src/node_options.h - --test-coverage-statements CLI option
doc/api/cli.md - Documentation for the new option
test/parallel/test-runner-coverage-statements.js - Tests

Test plan

Statement coverage fields appear in custom reporter JSON output
Coverage percentages are computed correctly (covered/uncovered statements)
--test-coverage-statements threshold passing and failing
Out-of-range threshold validation
stmts % column appears in tap/spec reporter output
100% coverage for fully covered files

Refs: #54530

Parse source files with acorn to extract AST statement nodes and map V8 coverage ranges to those statements, providing statement-level coverage metrics alongside existing line, branch, and function coverage. The implementation uses acorn-walk's Statement visitor to collect all non-BlockStatement nodes, then finds the most specific (smallest) V8 coverage range containing each statement to determine its execution count. Files that cannot be parsed by acorn gracefully degrade to 100% statement coverage. Adds --test-coverage-statements CLI option for setting minimum statement coverage thresholds, consistent with existing --test-coverage-lines, --test-coverage-branches, and --test-coverage-functions options. Refs: nodejs#54530

Review requested:

@nodejs/config
@nodejs/test_runner

How does this affect performance?

Good question. Here's the breakdown:

When coverage is disabled: zero overhead. The coverage module is lazy-loaded behind if (!globalOptions.coverage) in configureCoverage(), so acorn is never require'd.

When coverage IS enabled: getStatements() adds one acorn parse + walk per source file during summary(). The main new cost is the AST parse — getLines() already does a readFileSync per file, and getStatements() does another one (they cache independently via #sourceLines / #sourceStatements Maps, but both read the file). Worth noting that both reads are against the same file so the OS page cache makes the second read essentially free.

The parse results are cached per file URL, so repeated calls don't re-parse. Files that fail to parse gracefully degrade to null (100% statement coverage) rather than throwing.

Relative to V8's own coverage instrumentation overhead (which is already the cost of --experimental-test-coverage), one acorn parse per file at report time is negligible — it only runs once at the end, not during test execution.

One minor thing I noticed while looking at this: getStatements and getLines could potentially share the source read (pass the source from one to the other or cache at a shared level), but that's a small optimization and probably not worth the added coupling.

The acorn-walk `simple` function does not fire a generic "Statement" visitor for concrete statement node types, which meant the statements array was always empty and coveredStatementPercent was always 100%. Fix by enumerating each concrete statement type (ExpressionStatement, ReturnStatement, IfStatement, etc.) as individual visitor keys. Also fixes: - sourceType fallback: try 'module' first, catch and retry 'script' - Infinity primordial: replace bare Infinity with bestRange null pattern - double disk I/O: summary() now reads source once and passes it to both getLines() and getStatements() - tests: assert totalStatementCount > 0 to catch this regression

Need to re-review newer code

I don't like this level of hard coding, is there not a general Statement handler or something?

Good call — acorn-walk's simple() does support a generic Statement category visitor that fires for every node dispatched in a statement position. Replaced the hardcoded array with a single visitor.Statement handler and a small deny-set for BlockStatement/EmptyStatement.

This also picks up ClassDeclaration, FunctionDeclaration, and StaticBlock which were missing from the original list, and stays forward-compatible with any future ESTree statement types.

Simplified the double parse (module→script fallback) to a single sourceType: 'script' pass with permissive flags too, since script mode + allowImportExportEverywhere handles both ESM and legacy CJS.

- Replace hardcoded kStatementTypes array with acorn-walk's generic "Statement" category visitor, which automatically covers all current and future ESTree statement types (including ClassDeclaration, FunctionDeclaration, and StaticBlock that were previously missing). BlockStatement and EmptyStatement are excluded via a small deny-set. - Simplify AST parsing to a single pass using sourceType: 'script' with permissive flags (allowReturnOutsideFunction, allowImportExportEverywhere, allowAwaitOutsideFunction). Script mode is non-strict, so it handles both ESM and legacy CJS (e.g. `with` statements) without needing a module→script fallback. - Flatten V8 coverage ranges before the statement matching loop, reducing nesting from three levels to two. - Add coverage-class.js fixture and test for ClassDeclaration and StaticBlock statement coverage.

Haven’t had a chance to review yet, but conceptually, this is awesome - without all 4 standard coverage metrics, a testing solution is incomplete. Will review soon :-)

- Add allowHashBang to acorn parse options for shebang support - Reorder requires to follow ASCII convention - Reuse existing doesRangeContainOtherRange helper - Add comments to empty catch blocks for consistency - Remove else after return in findLineForOffset - Break kColumnsKeys across multiple lines (max-len) - Document statement coverage fields in test:coverage event schema - Migrate threshold tests to data-driven loop - Remove unused tmpdir import from test file - Add fixture proving statement != line coverage - Add fixture testing shebang file parsing - Add fixture testing graceful degradation for unparseable files Refs: nodejs#62340

For checking statement coverage thresholds this seems great! Does this also output coverage data that includes statement info?

You should create this object inline

Good call, inlined it!

Yes! Each file in the summary includes a statements array (with line and count per statement), plus totalStatementCount, coveredStatementCount, and coveredStatementPercent. The TAP/spec reporters also show a stmts % column.

Suggested change

      
                // if/for/while bodies, etc.). This automatically covers all current and
          
                // future ESTree statement types without hardcoding a list.
          
                // if/for/while bodies, etc.).

Codecov Report

❌ Patch coverage is 86.74033% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.68%. Comparing base (b328bf7) to head (e1cf6c7).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/test_runner/coverage.js	87.42%	21 Missing ⚠️
lib/internal/test_runner/test.js	0.00%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #62340    +/-   ##
========================================
  Coverage   89.68%   89.68%            
========================================
  Files         676      676            
  Lines      206689   206871   +182     
  Branches    39579    39603    +24     
========================================
+ Hits       185370   185542   +172     
- Misses      13450    13465    +15     
+ Partials     7869     7864     -5

Files with missing lines	Coverage Δ
lib/internal/test_runner/utils.js	62.59% <100.00%> (+0.39%)	⬆️
src/node_options.cc	76.37% <100.00%> (-0.08%)	⬇️
src/node_options.h	97.94% <100.00%> (+0.01%)	⬆️
lib/internal/test_runner/test.js	96.63% <0.00%> (-0.18%)	⬇️
lib/internal/test_runner/coverage.js	69.09% <87.42%> (+4.91%)	⬆️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

- Replace non-ASCII em dash with ASCII in coverage.js comments - Remove disallowed string literals from assert.strictEqual() 3rd arg - Inline visitor object per review feedback - Add --test-coverage-statements to doc/node.1 manpage - Update coverage report strings in test files to include stmts % column

Hey folks 👋

Pushed a fix commit addressing the lint issues, manpage entry, and updating the hardcoded report strings in the test files. However, I still need to regenerate the ~15 snapshot files in test/fixtures/test-runner/output/ that compare coverage table output.

I've been trying to build from source locally to run NODE_REGENERATE_SNAPSHOTS=1, but on Windows it's been quite the adventure:

WSL (Ubuntu 24.04): Build completed but the V8 snapshot generation step crashes with Fatal error in v8::ToLocalChecked - Empty MaybeLocal. Building with --without-node-snapshot produces a binary that segfaults when running the snapshot tests.
Docker Desktop: Build starts fine but the I/O overhead on Windows (overlay2 filesystem) makes it painfully slow — after ~60 minutes it had only compiled a fraction of the V8 objects.

If anyone has tips for building on Windows/WSL, or could trigger CI so I can grab the expected vs actual diffs from the test logs to update the snapshots manually, that would be really appreciated!

The changes pushed so far:

Fixed ESLint errors (non-ASCII em dash, assert.strictEqual 3rd arg)
Applied @avivkeller's comment suggestion
Added --test-coverage-statements to doc/node.1
Updated report strings in test-runner-coverage.js, test-runner-coverage-default-exclusion.mjs, test-runner-coverage-source-map.js, and test-runner-coverage-thresholds.js

cc: @aduh95 as this is the same person from #62337 (comment). It turns out they haven't even been building the project when submitting all of these PRs.

@Felipeness

When submitting a PR, we ask that you

Ensure that make -j4 test (UNIX), or vcbuild test (Windows) passes.

As you've said above, you have been unable to test this code. Given that, and:

Your other PRs to this project, which have gotten this same notice
Your PR description feels like a Claude plan
Your consistent use of — over easier to type alternatives, like -

We believe that you are, or are utilizing, an AI agent. In #62337, however, you stated that

I wouldn't submit something I don't understand.

However, if you've been unable to test the code, how can you be sure you understand it?

To quote @aduh95,

If you look like a bot, you risk being treated as such

At this point in time, you look like a bot.

Hmm.. since getStatements can return null if acorn cannot successfully parse the input, totalStatements here is going to be 0 and toPercentage will return 100, which doesn't seem right.

@Felipeness

At this point, you've opened several PRs. All of which have tell-tale signs of being completely AI agent generated. Your responses to feedback also appear to be almost entirely (if not entirely) AI-agent generated. While use of an AI-agent to assist in development is not problematic on it's own, we do expect that contributors understand and are capable of adhering to the contribution guidelines of the project. Across all of your PRs so far, every commit message fails to meet even the most basic guideline around the use of the subsystem {prefix}:.

As for technical issues with this PR are concerned, the generating 100% coverage for an unparseable file feels wrong; the lack of lazy loading for acorn in the coverage.js is an issue, and I while I didn't deep dive into the algorithms here I strongly suspect there's a pretty steep performance cliff lurking in here, particularly around the statement-to-range mapping. At this point I'd be -1 on landing this.

Fair point. Yes, I use AI agents as tools to assist my workflow, similar to how others use copilot or IDE extensions. The investigation, understanding, and decisions are mine. The agents help with execution.

About testing: I did build and run the project locally. On previous contributions I used a Mac and Linux where make -j4 test works without issues. This time I'm on a Windows machine, and building Node.js on Windows turned out to be significantly harder than I expected. ClangCL compiling v8_compiler.vcxproj causes LLVM ERROR: out of memory even with 32 GB of RAM. I had to research and increase the Windows page file to 48 GB just to get the build to complete. Even after that, the snapshot tests produce different binary output due to timing differences, so they can't be regenerated locally to match CI.

I mentioned this difficulty in my earlier comment and asked if anyone had tips for Windows contributors or could trigger CI so I could grab the diffs. That question was not addressed. Instead, the focus shifted to whether I use AI. I appreciate the concern around AI-generated contributions, and I understand why the team is cautious. But I'd also appreciate some help with the actual technical blocker I raised.

If there's a preferred workflow for Windows contributors to validate snapshot changes, I'm happy to follow it.

And honestly, I don't mind being treated like a bot. That's fine.

But I do think code reviews could be more fun. I thought this would be a space to exchange ideas in a playful way, like "this code is so bad it shouldn't even exist, but try harder, you're on the right track." That's the kind of thing friends do because they're friends. That's the energy I was hoping for when I started contributing here.

@Felipeness ... no worries. The thing that makes us nervous is when all work and all comments from a user are obviously all AI-agent generated. It doesn't make it fun for the rest of us ;-) ... we need to know there's an actual person who understands the changes they're are making on the other side. The way trust is built is by being authentic. Definitely good to have fun with things but if the only thing contributors see of you are your AI-agent outputs they're going to tune out and ignore your contributions very quickly.

Thanks @jasnell. If it's not fun for you it's not fun for me either, that's exactly why I'm here.

The robotic tone comes from English not being my native language. I write in bad English, ask AI to clean up the grammar and it comes out sounding like a machine wrote everything. What matters to me is learning to communicate better in English so the process is I write, AI comments, I rewrite. Looks robotic but I'm learning a lot from it. And honestly I'm a bit robotic even in Portuguese, I'm autistic so that's just how I sound. But make no mistake you're talking to someone who has the capacity and does not delegate capacity to AI, that doesn't even make sense to me.

On the performance point you raised about statement-to-range mapping, that's something I don't know yet. I'll dig into it.

- lazy-load acorn and acorn-walk behind ??= guard - fix 100% statement coverage for unparseable files - sort coverage ranges by startOffset for early loop exit - regenerate snapshot files from native linux build - update hardcoded report strings in coverage tests

To be honest, I would prefer to read the authentic bad English than the version rewritten by AI. If the AI can understand you, we probably can too.

When statement coverage is present you could derive proper line coverage from it like istanbuljs does.

This would resolve #60996.

To be honest, I would prefer to read the authentic bad English than the version rewritten by AI. If the AI can understand you, we probably can too.

No problem then, now i write everything down

use lazy from util

This amuses me just due to the fact that the V8/node test coverage concept was predicated on not having to instrument the code before running it. But because it produced such incredibly inaccurate results we're now effectively doing post-coverage instrumentation by parsing source code and mapping the extremely inaccurate coverage results back to a proper AST.

Now add proper implicit branch coverage!

This pull request has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

This was closed while I was away, so I understand if it stays closed. Posting what I have in case it is useful.

@jasnell the three points you raised are addressed on the branch:

100% coverage on unparseable files: fixed, reports 0. A file with no measurable statements is not a covered file.
acorn not lazy loaded: fixed, uses getLazy() from internal/util, same pattern as internal/fs/promises.
statement to range mapping: ranges sorted by startOffset so the loop exits early.

@avivkeller getLazy applied.

Two tests were still asserting the old 100% and were red. Fixed both: the unparseable case in test-runner-coverage-statements.js, and stdin.test.ts in test-runner-coverage-source-map.js, which is a source map without the original file.

Verified on Linux: coverage suite 5/5, full parallel suite 4031/4032. The single failure is test-setproctitle.js, which asserts stderr is empty while my terminal writes a tty warning to it.

On the build: my earlier PRs were done on a MacBook. I moved to Windows around this one and that is where I got stuck. The causes were the build dying with SIGHUP when the launching shell exits, and running from /mnt/c over 9p. Native ext4 plus ninja instead of make gives a 55 second incremental rebuild. Wrote it down in case it saves someone else the time: https://gist.github.com/Felipeness/a3e10794f996cbdffb6cee2313a999c4

The branch needs a rebase against main. Happy to do that and reopen if there is interest in the feature, and equally happy to leave it closed.

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Mar 19, 2026

avivkeller previously approved these changes Mar 19, 2026

View reviewed changes

avivkeller reviewed Mar 19, 2026

View reviewed changes

ljharb reviewed Mar 20, 2026

View reviewed changes

pmarchini reviewed Mar 20, 2026

View reviewed changes

Comment thread test/parallel/test-runner-coverage-statements.js

avivkeller reviewed Mar 20, 2026

View reviewed changes

refactor: inline visitor object in getStatements

e1cf6c7

avivkeller reviewed Mar 22, 2026

View reviewed changes

jasnell reviewed Mar 22, 2026

View reviewed changes

Felipeness mentioned this pull request Mar 23, 2026

src: skip JS callback for settled Promise.race losers #62336

Closed

AriPerkkio reviewed Mar 27, 2026

View reviewed changes

avivkeller reviewed Apr 8, 2026

View reviewed changes

mcollina mentioned this pull request May 23, 2026

test_runner: tracking issue for code coverage stabilization #63506

Open

31 tasks

coderabbitai Bot mentioned this pull request Jul 3, 2026

Add NODE0-SPINE-RUNNER-CLI-1A BizraInfo/Dema#312

Merged

5 tasks

github-actions Bot added the stale label Aug 16, 2026

avivkeller closed this Aug 17, 2026

+              const kStatementTypes = [
+                'ExpressionStatement', 'ReturnStatement', 'ThrowStatement',
+                'IfStatement', 'WhileStatement', 'DoWhileStatement',
+                'ForStatement', 'ForInStatement', 'ForOfStatement',
+                'SwitchStatement', 'TryStatement', 'BreakStatement',
+                'ContinueStatement', 'VariableDeclaration', 'LabeledStatement',
+                'WithStatement', 'DebuggerStatement',

+                  // acorn-walk's simple() fires a generic "Statement" visitor for every
+                  // node dispatched in a statement position (Program body, block bodies,
+                  // if/for/while bodies, etc.). This automatically covers all current and
+                  // future ESTree statement types without hardcoding a list.
+                  const visitor = { __proto__: null };
+                  visitor.Statement = (node) => {
+                    if (kExcludedStatementTypes.has(node.type)) return;
+                    ArrayPrototypePush(statements, {
+                      __proto__: null,
+                      startOffset: node.start,
+                      endOffset: node.end,
+                      count: 0,
+                    });
+                  };

                       coveredLinePercent: toPercentage(coveredCnt, lines.length),
                       coveredBranchPercent: toPercentage(branchesCovered, totalBranches),
                       coveredFunctionPercent: toPercentage(functionsCovered, totalFunctions),
+                      coveredStatementPercent: toPercentage(statementsCovered, totalStatements),

               const { fileURLToPath, URL } = require('internal/url');
               const { kMappings, SourceMap } = require('internal/source_map/source_map');
+              let AcornParser;  // Lazy loaded -- only needed when statement coverage is enabled.
+              let acornWalkSimple;

		// if/for/while bodies, etc.). This automatically covers all current and
		// future ESTree statement types without hardcoding a list.

	// if/for/while bodies, etc.). This automatically covers all current and
	// future ESTree statement types without hardcoding a list.
	// if/for/while bodies, etc.).

test_runner: add statement coverage support by Felipeness · Pull Request #62340 · nodejs/node · GitHub

Uh oh!

Uh oh!

test_runner: add statement coverage support - #62340

test_runner: add statement coverage support#62340Felipeness wants to merge 7 commits intonodejs:mainnodejs/node:mainfrom Felipeness:feat/test-runner-statement-coverageFelipeness/node:feat/test-runner-statement-coverageCopy head branch name to clipboard

Conversation

Felipeness commented Mar 19, 2026

Summary

Implementation

Files changed

Test plan

Uh oh!

nodejs-github-bot commented Mar 19, 2026

Uh oh!

avivkeller commented Mar 19, 2026

Uh oh!

Felipeness commented Mar 19, 2026

Uh oh!

avivkeller Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

Felipeness Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

ljharb commented Mar 20, 2026

Uh oh!

ljharb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

avivkeller Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Felipeness Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Felipeness commented Mar 21, 2026

Uh oh!

avivkeller Mar 22, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Felipeness commented Mar 22, 2026

Uh oh!

cjihrig commented Mar 22, 2026

Uh oh!

avivkeller commented Mar 22, 2026

Uh oh!

jasnell Mar 22, 2026

Choose a reason for hiding this comment

Uh oh!

jasnell commented Mar 22, 2026

Uh oh!

Felipeness commented Mar 23, 2026

Uh oh!

Felipeness commented Mar 23, 2026

Uh oh!

jasnell commented Mar 23, 2026

Uh oh!

Felipeness commented Mar 23, 2026

Uh oh!

targos commented Mar 26, 2026

Uh oh!

AriPerkkio left a comment

Choose a reason for hiding this comment

Uh oh!

Felipeness commented Mar 27, 2026

Uh oh!

avivkeller Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

jcready commented May 17, 2026

Uh oh!

github-actions Bot commented Aug 16, 2026

Uh oh!

Felipeness commented Aug 20, 2026

Uh oh!

test_runner: add statement coverage support#62340
Felipeness wants to merge 7 commits into
nodejs:mainfrom
Felipeness:feat/test-runner-statement-coverage

codecov Bot commented Mar 22, 2026 •
edited
Loading