FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

test_runner: add tags option and tag-name filter · nodejs/node@d03d968 · GitHub

/ node Public

Commit d03d968

Browse files
atlowChemi
authored and
aduh95
committed
test_runner: add tags option and tag-name filter
Adds a `tags` option to test(), it(), suite(), and describe() that accepts an array of string labels. Tags are canonicalized to lowercase and inherited from suites to nested tests by union. Reporter events expose the tag set on every test, and TestContext exposes the test's tags via `context.tags`. Filtering is done by literal tag name through the new `--experimental-test-tag-filter=<tag>` flag (or `testTagFilters` on run()). The flag may be specified more than once; tests must contain every filter to run. Untagged tests are excluded under any positive filter. The tagging mechanism is gated behind a one-shot ExperimentalWarning. Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63221 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Pietro Marchini <pietro.marchini94@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh>
1 parent e8c3db1 commit d03d968

17 files changed

Lines changed: 981 additions & 16 deletions

‎doc/api/cli.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1237,6 +1237,23 @@ Enable module mocking in the test runner.
12371237

12381238
This feature requires `--allow-worker` if used with the [Permission Model][].
12391239

1240+
### `--experimental-test-tag-filter=<tag>`
1241+
1242+
<!-- YAML
1243+
added: REPLACEME
1244+
-->
1245+
1246+
> Stability: 1.0 - Early development
1247+
1248+
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1249+
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1250+
inherit from suites to nested tests by union. Filtering is
1251+
case-insensitive.
1252+
1253+
The flag may be specified more than once; tests must contain **every**
1254+
filter value to run. See [Test tags][] for details on declaring and
1255+
inheriting tags.
1256+
12401257
### `--experimental-transform-types`
12411258

12421259
<!-- YAML
@@ -4140,6 +4157,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
41404157
[ScriptCoverage]: https://chromedevtools.github.io/devtools-protocol/tot/Profiler#type-ScriptCoverage
41414158
[ShadowRealm]: https://github.com/tc39/proposal-shadowrealm
41424159
[Source Map]: https://tc39.es/ecma426/
4160+
[Test tags]: test.md#test-tags
41434161
[TypeScript type-stripping]: typescript.md#type-stripping
41444162
[V8 Inspector integration for Node.js]: debugger.md#v8-inspector-integration-for-nodejs
41454163
[V8 JavaScript code coverage]: https://v8project.blogspot.com/2017/12/javascript-code-coverage.html

‎doc/api/test.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -478,6 +478,82 @@ Test name patterns do not change the set of files that the test runner executes.
478478
If both `--test-name-pattern` and `--test-skip-pattern` are supplied,
479479
tests must satisfy **both** requirements in order to be executed.
480480

481+
## Test tags
482+
483+
<!-- YAML
484+
added: REPLACEME
485+
-->
486+
487+
> Stability: 1.0 - Early development
488+
489+
Tags annotate tests and suites with arbitrary string labels. The
490+
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
491+
option on [`run()`][]) selects tests whose tag set contains every
492+
provided filter value.
493+
494+
Tags are an alternative to encoding metadata into test names. They are
495+
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
496+
or environment, where a name pattern would be brittle.
497+
498+
### Authoring tagged tests
499+
500+
Pass a `tags` array on any of `test()`, `it()`, `suite()`, or `describe()`.
501+
Tags inherit from a suite to its child tests by union—a test inside a
502+
suite tagged `['db']` that declares its own `tags: ['integration']`
503+
effectively has both tags.
504+
505+
```mjs
506+
import { describe, it } from 'node:test';
507+
508+
describe('database', { tags: ['db'] }, () => {
509+
it('reads a row'); // tags: ['db']
510+
it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration']
511+
it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky']
512+
});
513+
```
514+
515+
```cjs
516+
const { describe, it } = require('node:test');
517+
518+
describe('database', { tags: ['db'] }, () => {
519+
it('reads a row'); // tags: ['db']
520+
it('writes a row', { tags: ['integration'] }); // tags: ['db', 'integration']
521+
it('reconnects after disconnect', { tags: ['flaky'] }); // tags: ['db', 'flaky']
522+
});
523+
```
524+
525+
Tag values must be non-empty strings. Tags are matched case-insensitively;
526+
the canonical form is lowercase. Duplicates within a single `tags` array
527+
are collapsed on the lowercased form, preserving the first-seen
528+
declaration order.
529+
530+
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
531+
own tags. They run as part of their owning suite, which carries the
532+
suite's tags.
533+
534+
### Filtering by tag
535+
536+
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
537+
test runs only when its tag set contains that name. The flag may be
538+
specified more than once; tests must match **every** filter to run. The
539+
same applies to the `testTagFilters` array on [`run()`][]. Filters are
540+
case-insensitive and AND'd with [`--test-name-pattern`][],
541+
[`--test-skip-pattern`][], and `.only` filtering.
542+
543+
Untagged tests are excluded under any non-empty filter, since the filter
544+
requires the tag to be present.
545+
546+
### Reading tags from inside a test
547+
548+
The [`TestContext`][] object exposes the test's tags as a frozen array
549+
through [`context.tags`][], so tests can branch on their own metadata.
550+
551+
### Errors
552+
553+
A tag value that violates the validation rules above throws
554+
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
555+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
556+
481557
## Extraneous asynchronous activity
482558

483559
Once a test function finishes executing, the results are reported as quickly
@@ -749,6 +825,8 @@ test runner functionality:
749825

750826
* `--test` - Prevented to avoid recursive test execution
751827
* `--experimental-test-coverage` - Managed by the test runner
828+
* `--experimental-test-tag-filter` - Filter values are validated by the parent
829+
process and re-emitted to child processes
752830
* `--watch` - Watch mode is handled at the parent level
753831
* `--experimental-default-config-file` - Config file loading is handled by the parent
754832
* `--test-reporter` - Reporting is managed by the parent process
@@ -1570,6 +1648,9 @@ added:
15701648
- v18.9.0
15711649
- v16.19.0
15721650
changes:
1651+
- version: REPLACEME
1652+
pr-url: https://github.com/nodejs/node/pull/63221
1653+
description: Added the `testTagFilters` option.
15731654
- version: v24.14.0
15741655
pr-url: https://github.com/nodejs/node/pull/61367
15751656
description: Add the `env` option.
@@ -1656,6 +1737,10 @@ changes:
16561737
For each test that is executed, any corresponding test hooks, such as
16571738
`beforeEach()`, are also run.
16581739
**Default:** `undefined`.
1740+
* `testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1741+
used to filter tests by their declared tags. Tests must contain every
1742+
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1743+
on the command line. See [Test tags][]. **Default:** `undefined`.
16591744
* `timeout` {number} A number of milliseconds the test execution will
16601745
fail after.
16611746
If unspecified, subtests inherit this value from their parent.
@@ -1799,6 +1884,9 @@ added:
17991884
- v18.0.0
18001885
- v16.17.0
18011886
changes:
1887+
- version: REPLACEME
1888+
pr-url: https://github.com/nodejs/node/pull/63221
1889+
description: Added the `tags` option.
18021890
- version:
18031891
- v20.2.0
18041892
- v18.17.0
@@ -1842,6 +1930,10 @@ changes:
18421930
* `skip` {boolean|string} If truthy, the test is skipped. If a string is
18431931
provided, that string is displayed in the test results as the reason for
18441932
skipping the test. **Default:** `false`.
1933+
* `tags` {string\[]} An array of string labels associated with the test.
1934+
Used together with [`--experimental-test-tag-filter`][] to filter which
1935+
tests run. Tags inherit from suites to nested tests by union. See
1936+
[Test tags][]. **Default:** `[]`.
18451937
* `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string
18461938
is provided, that string is displayed in the test results as the reason why
18471939
the test is `TODO`. **Default:** `false`.
@@ -3425,6 +3517,9 @@ Emitted when code coverage is enabled and all tests have completed.
34253517
`undefined` if the test was run through the REPL.
34263518
* `name` {string} The test name.
34273519
* `nesting` {number} The nesting level of the test.
3520+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3521+
and its ancestor suites, in declaration order. Empty for untagged tests.
3522+
See [Test tags][].
34283523
* `testId` {number} A numeric identifier for this test instance, unique
34293524
within the test file's process. Consistent across all events for the same
34303525
test instance, enabling reliable correlation in custom reporters.
@@ -3448,6 +3543,9 @@ The corresponding declaration ordered events are `'test:pass'` and `'test:fail'`
34483543
`undefined` if the test was run through the REPL.
34493544
* `name` {string} The test name.
34503545
* `nesting` {number} The nesting level of the test.
3546+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3547+
and its ancestor suites, in declaration order. Empty for untagged tests.
3548+
See [Test tags][].
34513549
* `testId` {number} A numeric identifier for this test instance, unique
34523550
within the test file's process. Consistent across all events for the same
34533551
test instance, enabling reliable correlation in custom reporters.
@@ -3489,6 +3587,9 @@ defined.
34893587
`undefined` if the test was run through the REPL.
34903588
* `name` {string} The test name.
34913589
* `nesting` {number} The nesting level of the test.
3590+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3591+
and its ancestor suites, in declaration order. Empty for untagged tests.
3592+
See [Test tags][].
34923593
* `testId` {number} A numeric identifier for this test instance, unique
34933594
within the test file's process. Consistent across all events for the same
34943595
test instance, enabling reliable correlation in custom reporters.
@@ -3515,6 +3616,9 @@ Emitted when a test is enqueued for execution.
35153616
`undefined` if the test was run through the REPL.
35163617
* `name` {string} The test name.
35173618
* `nesting` {number} The nesting level of the test.
3619+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3620+
and its ancestor suites, in declaration order. Empty for untagged tests.
3621+
See [Test tags][].
35183622
* `testId` {number} A numeric identifier for this test instance, unique
35193623
within the test file's process. Consistent across all events for the same
35203624
test instance, enabling reliable correlation in custom reporters.
@@ -3572,6 +3676,9 @@ since the parent runner only knows about file-level tests. When using
35723676
`undefined` if the test was run through the REPL.
35733677
* `name` {string} The test name.
35743678
* `nesting` {number} The nesting level of the test.
3679+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3680+
and its ancestor suites, in declaration order. Empty for untagged tests.
3681+
See [Test tags][].
35753682
* `testId` {number} A numeric identifier for this test instance, unique
35763683
within the test file's process. Consistent across all events for the same
35773684
test instance, enabling reliable correlation in custom reporters.
@@ -3611,6 +3718,9 @@ defined.
36113718
`undefined` if the test was run through the REPL.
36123719
* `name` {string} The test name.
36133720
* `nesting` {number} The nesting level of the test.
3721+
* `tags` {string\[]} The flattened lowercased tags declared on the test
3722+
and its ancestor suites, in declaration order. Empty for untagged tests.
3723+
See [Test tags][].
36143724
* `testId` {number} A numeric identifier for this test instance, unique
36153725
within the test file's process. Consistent across all events for the same
36163726
test instance, enabling reliable correlation in custom reporters.
@@ -4114,6 +4224,20 @@ The attempt number of the test. This value is zero-based, so the first attempt i
41144224
the second attempt is `1`, and so on. This property is useful in conjunction with the
41154225
`--test-rerun-failures` option to determine which attempt the test is currently running.
41164226

4227+
### `context.tags`
4228+
4229+
<!-- YAML
4230+
added: REPLACEME
4231+
-->
4232+
4233+
> Stability: 1.0 - Early development
4234+
4235+
* Type: {string\[]}
4236+
4237+
A frozen array of the test's flattened lowercased tags, in declaration
4238+
order, including any tags inherited from ancestor suites. Empty when the
4239+
test has no tags. See [Test tags][].
4240+
41174241
### `context.workerId`
41184242

41194243
<!-- YAML
@@ -4329,6 +4453,9 @@ added:
43294453
- v18.0.0
43304454
- v16.17.0
43314455
changes:
4456+
- version: REPLACEME
4457+
pr-url: https://github.com/nodejs/node/pull/63221
4458+
description: Added the `tags` option.
43324459
- version:
43334460
- v18.8.0
43344461
- v16.18.0
@@ -4359,6 +4486,10 @@ changes:
43594486
* `skip` {boolean|string} If truthy, the test is skipped. If a string is
43604487
provided, that string is displayed in the test results as the reason for
43614488
skipping the test. **Default:** `false`.
4489+
* `tags` {string\[]} An array of string labels associated with the subtest.
4490+
Used together with [`--experimental-test-tag-filter`][] to filter which
4491+
tests run. Tags inherit from the parent test or suite by union. See
4492+
[Test tags][]. **Default:** `[]`.
43624493
* `todo` {boolean|string} If truthy, the test marked as `TODO`. If a string
43634494
is provided, that string is displayed in the test results as the reason why
43644495
the test is `TODO`. **Default:** `false`.
@@ -4507,8 +4638,10 @@ test.describe('my suite', (suite) => {
45074638
```
45084639

45094640
[TAP]: https://testanything.org/
4641+
[Test tags]: #test-tags
45104642
[`--experimental-test-coverage`]: cli.md#--experimental-test-coverage
45114643
[`--experimental-test-module-mocks`]: cli.md#--experimental-test-module-mocks
4644+
[`--experimental-test-tag-filter`]: cli.md#--experimental-test-tag-filtertag
45124645
[`--import`]: cli.md#--importmodule
45134646
[`--no-strip-types`]: cli.md#--no-strip-types
45144647
[`--test-concurrency`]: cli.md#--test-concurrency
@@ -4534,6 +4667,7 @@ test.describe('my suite', (suite) => {
45344667
[`assert.throws`]: assert.md#assertthrowsfn-error-message
45354668
[`context.diagnostic`]: #contextdiagnosticmessage
45364669
[`context.skip`]: #contextskipmessage
4670+
[`context.tags`]: #contexttags
45374671
[`context.todo`]: #contexttodomessage
45384672
[`describe()`]: #describename-options-fn
45394673
[`diagnostics_channel`]: diagnostics_channel.md

‎doc/node.1‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,10 @@ Disable support for loading ECMAScript modules with require().
222222
.It Fl -no-strip-types
223223
Disable type-stripping for TypeScript files.
224224
.
225+
.It Fl -experimental-test-tag-filter Ar tag
226+
Run only tests whose tag set contains \fItag\fR. May be specified multiple
227+
times; tests must contain every filter to run.
228+
.
225229
.It Fl -experimental-vm-modules
226230
Enable experimental ES module support in VM module.
227231
.

‎lib/internal/test_runner/harness.js‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ function createTestTree(rootTestOptions, globalOptions) {
4747
globalOptions.testSkipPatterns;
4848
const isFilteringByOnly = (globalOptions.isolation === 'process' || process.env.NODE_TEST_CONTEXT) ?
4949
globalOptions.only : true;
50+
const isFilteringByTags = globalOptions.testTagFilters != null;
5051
const harness = {
5152
__proto__: null,
5253
buildPromise: buildPhaseDeferred.promise,
@@ -76,6 +77,7 @@ function createTestTree(rootTestOptions, globalOptions) {
7677
previousRuns: null,
7778
isFilteringByName,
7879
isFilteringByOnly,
80+
isFilteringByTags,
7981
async runBootstrap() {
8082
if (globalSetupExecuted) {
8183
return PromiseResolve();

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL