FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

test_runner: add --test-name-pattern CLI flag · nodejs/node@d762a34 · GitHub

/ node Public

Commit d762a34

Browse files
cjihrig
authored and
danielleadams
committed
test_runner: add --test-name-pattern CLI flag
This commit adds support for running tests that match a regular expression. Fixes: #42984
1 parent e92b074 commit d762a34

12 files changed

Lines changed: 333 additions & 4 deletions

‎doc/api/cli.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1187,6 +1187,16 @@ Starts the Node.js command line test runner. This flag cannot be combined with
11871187
`--check`, `--eval`, `--interactive`, or the inspector. See the documentation
11881188
on [running tests from the command line][] for more details.
11891189

1190+
### `--test-name-pattern`
1191+
1192+
<!-- YAML
1193+
added: REPLACEME
1194+
-->
1195+
1196+
A regular expression that configures the test runner to only execute tests
1197+
whose name matches the provided pattern. See the documentation on
1198+
[filtering tests by name][] for more details.
1199+
11901200
### `--test-only`
11911201

11921202
<!-- YAML
@@ -2262,6 +2272,7 @@ done
22622272
[debugger]: debugger.md
22632273
[debugging security implications]: https://nodejs.org/en/docs/guides/debugging-getting-started/#security-implications
22642274
[emit_warning]: process.md#processemitwarningwarning-options
2275+
[filtering tests by name]: test.md#filtering-tests-by-name
22652276
[jitless]: https://v8.dev/blog/jitless
22662277
[libuv threadpool documentation]: https://docs.libuv.org/en/latest/threadpool.html
22672278
[remote code execution]: https://www.owasp.org/index.php/Code_Injection

‎doc/api/test.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,42 @@ test('this test is not run', () => {
220220
});
221221
```
222222

223+
## Filtering tests by name
224+
225+
The [`--test-name-pattern`][] command-line option can be used to only run tests
226+
whose name matches the provided pattern. Test name patterns are interpreted as
227+
JavaScript regular expressions. The `--test-name-pattern` option can be
228+
specified multiple times in order to run nested tests. For each test that is
229+
executed, any corresponding test hooks, such as `beforeEach()`, are also
230+
run.
231+
232+
Given the following test file, starting Node.js with the
233+
`--test-name-pattern="test [1-3]"` option would cause the test runner to execute
234+
`test 1`, `test 2`, and `test 3`. If `test 1` did not match the test name
235+
pattern, then its subtests would not execute, despite matching the pattern. The
236+
same set of tests could also be executed by passing `--test-name-pattern`
237+
multiple times (e.g. `--test-name-pattern="test 1"`,
238+
`--test-name-pattern="test 2"`, etc.).
239+
240+
```js
241+
test('test 1', async (t) => {
242+
await t.test('test 2');
243+
await t.test('test 3');
244+
});
245+
246+
test('Test 4', async (t) => {
247+
await t.test('Test 5');
248+
await t.test('test 6');
249+
});
250+
```
251+
252+
Test name patterns can also be specified using regular expression literals. This
253+
allows regular expression flags to be used. In the previous example, starting
254+
Node.js with `--test-name-pattern="/test [4-5]/i"` would match `Test 4` and
255+
`Test 5` because the pattern is case-insensitive.
256+
257+
Test name patterns do not change the set of files that the test runner executes.
258+
223259
## Extraneous asynchronous activity
224260

225261
Once a test function finishes executing, the TAP results are output as quickly
@@ -896,6 +932,7 @@ added: v18.7.0
896932
aborted.
897933

898934
[TAP]: https://testanything.org/
935+
[`--test-name-pattern`]: cli.md#--test-name-pattern
899936
[`--test-only`]: cli.md#--test-only
900937
[`--test`]: cli.md#--test
901938
[`SuiteContext`]: #class-suitecontext

‎doc/node.1‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,6 +387,10 @@ Specify the minimum allocation from the OpenSSL secure heap. The default is 2. T
387387
.It Fl -test
388388
Starts the Node.js command line test runner.
389389
.
390+
.It Fl -test-name-pattern
391+
A regular expression that configures the test runner to only execute tests
392+
whose name matches the provided pattern.
393+
.
390394
.It Fl -test-only
391395
Configures the test runner to only execute top level tests that have the `only`
392396
option set.

‎lib/internal/test_runner/test.js‎

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
'use strict';
22
const {
3+
ArrayPrototypeMap,
34
ArrayPrototypePush,
45
ArrayPrototypeReduce,
56
ArrayPrototypeShift,
67
ArrayPrototypeSlice,
8+
ArrayPrototypeSome,
79
ArrayPrototypeUnshift,
810
FunctionPrototype,
911
MathMax,
@@ -12,6 +14,7 @@ const {
1214
PromisePrototypeThen,
1315
PromiseResolve,
1416
ReflectApply,
17+
RegExpPrototypeExec,
1518
SafeMap,
1619
SafeSet,
1720
SafePromiseAll,
@@ -30,7 +33,11 @@ const {
3033
} = require('internal/errors');
3134
const { getOptionValue } = require('internal/options');
3235
const { TapStream } = require('internal/test_runner/tap_stream');
33-
const { createDeferredCallback, isTestFailureError } = require('internal/test_runner/utils');
36+
const {
37+
convertStringToRegExp,
38+
createDeferredCallback,
39+
isTestFailureError,
40+
} = require('internal/test_runner/utils');
3441
const {
3542
createDeferredPromise,
3643
kEmptyObject,
@@ -58,6 +65,13 @@ const kDefaultTimeout = null;
5865
const noop = FunctionPrototype;
5966
const isTestRunner = getOptionValue('--test');
6067
const testOnlyFlag = !isTestRunner && getOptionValue('--test-only');
68+
const testNamePatternFlag = isTestRunner ? null :
69+
getOptionValue('--test-name-pattern');
70+
const testNamePatterns = testNamePatternFlag?.length > 0 ?
71+
ArrayPrototypeMap(
72+
testNamePatternFlag,
73+
(re) => convertStringToRegExp(re, '--test-name-pattern')
74+
) : null;
6175
const kShouldAbort = Symbol('kShouldAbort');
6276
const kRunHook = Symbol('kRunHook');
6377
const kHookNames = ObjectSeal(['before', 'after', 'beforeEach', 'afterEach']);
@@ -195,6 +209,18 @@ class Test extends AsyncResource {
195209
this.timeout = timeout;
196210
}
197211

212+
if (testNamePatterns !== null) {
213+
// eslint-disable-next-line no-use-before-define
214+
const match = this instanceof TestHook || ArrayPrototypeSome(
215+
testNamePatterns,
216+
(re) => RegExpPrototypeExec(re, name) !== null
217+
);
218+
219+
if (!match) {
220+
skip = 'test name does not match pattern';
221+
}
222+
}
223+
198224
if (testOnlyFlag && !this.only) {
199225
skip = '\'only\' option not set';
200226
}
@@ -210,7 +236,6 @@ class Test extends AsyncResource {
210236
validateAbortSignal(signal, 'options.signal');
211237
this.#outerSignal?.addEventListener('abort', this.#abortHandler);
212238

213-
214239
this.fn = fn;
215240
this.name = name;
216241
this.parent = parent;
@@ -669,6 +694,7 @@ class ItTest extends Test {
669694
return { ctx: { signal: this.signal, name: this.name }, args: [] };
670695
}
671696
}
697+
672698
class Suite extends Test {
673699
constructor(options) {
674700
super(options);
@@ -704,7 +730,6 @@ class Suite extends Test {
704730
return;
705731
}
706732

707-
708733
const hookArgs = this.getRunArgs();
709734
await this[kRunHook]('before', hookArgs);
710735
const stopPromise = stopTest(this.timeout, this.signal);

‎lib/internal/test_runner/utils.js‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
'use strict';
2-
const { RegExpPrototypeExec } = primordials;
2+
const { RegExp, RegExpPrototypeExec } = primordials;
33
const { basename } = require('path');
44
const { createDeferredPromise } = require('internal/util');
55
const {
66
codes: {
7+
ERR_INVALID_ARG_VALUE,
78
ERR_TEST_FAILURE,
89
},
910
kIsNodeError,
1011
} = require('internal/errors');
1112

1213
const kMultipleCallbackInvocations = 'multipleCallbackInvocations';
14+
const kRegExpPattern = /^\/(.*)\/([a-z]*)$/;
1315
const kSupportedFileExtensions = /\.[cm]?js$/;
1416
const kTestFilePattern = /((^test(-.+)?)|(.+[.\-_]test))\.[cm]?js$/;
1517

@@ -54,7 +56,26 @@ function isTestFailureError(err) {
5456
return err?.code === 'ERR_TEST_FAILURE' && kIsNodeError in err;
5557
}
5658

59+
function convertStringToRegExp(str, name) {
60+
const match = RegExpPrototypeExec(kRegExpPattern, str);
61+
const pattern = match?.[1] ?? str;
62+
const flags = match?.[2] || '';
63+
64+
try {
65+
return new RegExp(pattern, flags);
66+
} catch (err) {
67+
const msg = err?.message;
68+
69+
throw new ERR_INVALID_ARG_VALUE(
70+
name,
71+
str,
72+
`is an invalid regular expression.${msg ? ` ${msg}` : ''}`
73+
);
74+
}
75+
}
76+
5777
module.exports = {
78+
convertStringToRegExp,
5879
createDeferredCallback,
5980
doesPathMatchFilter,
6081
isSupportedFileType,

‎src/node_options.cc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -553,6 +553,9 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
553553
AddOption("--test",
554554
"launch test runner on startup",
555555
&EnvironmentOptions::test_runner);
556+
AddOption("--test-name-pattern",
557+
"run tests whose name matches this regular expression",
558+
&EnvironmentOptions::test_name_pattern);
556559
AddOption("--test-only",
557560
"run tests with 'only' option set",
558561
&EnvironmentOptions::test_only,

‎src/node_options.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@ class EnvironmentOptions : public Options {
153153
std::string redirect_warnings;
154154
std::string diagnostic_dir;
155155
bool test_runner = false;
156+
std::vector<std::string> test_name_pattern;
156157
bool test_only = false;
157158
bool test_udp_no_try_send = false;
158159
bool throw_deprecation = false;

‎test/message/test_runner_test_name_pattern.js‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
// Flags: --no-warnings --test-name-pattern=enabled --test-name-pattern=/pattern/i
2+
'use strict';
3+
const common = require('../common');
4+
const {
5+
after,
6+
afterEach,
7+
before,
8+
beforeEach,
9+
describe,
10+
it,
11+
test,
12+
} = require('node:test');
13+
14+
test('top level test disabled', common.mustNotCall());
15+
test('top level skipped test disabled', { skip: true }, common.mustNotCall());
16+
test('top level skipped test enabled', { skip: true }, common.mustNotCall());
17+
it('top level it enabled', common.mustCall());
18+
it('top level it disabled', common.mustNotCall());
19+
it.skip('top level skipped it disabled', common.mustNotCall());
20+
it.skip('top level skipped it enabled', common.mustNotCall());
21+
describe('top level describe disabled', common.mustNotCall());
22+
describe.skip('top level skipped describe disabled', common.mustNotCall());
23+
describe.skip('top level skipped describe enabled', common.mustNotCall());
24+
test('top level runs because name includes PaTtErN', common.mustCall());
25+
26+
test('top level test enabled', common.mustCall(async (t) => {
27+
t.beforeEach(common.mustCall());
28+
t.afterEach(common.mustCall());
29+
await t.test(
30+
'nested test runs because name includes PATTERN',
31+
common.mustCall()
32+
);
33+
}));
34+
35+
describe('top level describe enabled', () => {
36+
before(common.mustCall());
37+
beforeEach(common.mustCall(2));
38+
afterEach(common.mustCall(2));
39+
after(common.mustCall());
40+
41+
it('nested it disabled', common.mustNotCall());
42+
it('nested it enabled', common.mustCall());
43+
describe('nested describe disabled', common.mustNotCall());
44+
describe('nested describe enabled', common.mustCall(() => {
45+
it('is enabled', common.mustCall());
46+
}));
47+
});

‎test/message/test_runner_test_name_pattern.out‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
TAP version 13
2+
# Subtest: top level test disabled
3+
ok 1 - top level test disabled # SKIP test name does not match pattern
4+
---
5+
duration_ms: *
6+
...
7+
# Subtest: top level skipped test disabled
8+
ok 2 - top level skipped test disabled # SKIP test name does not match pattern
9+
---
10+
duration_ms: *
11+
...
12+
# Subtest: top level skipped test enabled
13+
ok 3 - top level skipped test enabled # SKIP
14+
---
15+
duration_ms: *
16+
...
17+
# Subtest: top level it enabled
18+
ok 4 - top level it enabled
19+
---
20+
duration_ms: *
21+
...
22+
# Subtest: top level it disabled
23+
ok 5 - top level it disabled # SKIP test name does not match pattern
24+
---
25+
duration_ms: *
26+
...
27+
# Subtest: top level skipped it disabled
28+
ok 6 - top level skipped it disabled # SKIP test name does not match pattern
29+
---
30+
duration_ms: *
31+
...
32+
# Subtest: top level skipped it enabled
33+
ok 7 - top level skipped it enabled # SKIP
34+
---
35+
duration_ms: *
36+
...
37+
# Subtest: top level describe disabled
38+
ok 8 - top level describe disabled # SKIP test name does not match pattern
39+
---
40+
duration_ms: *
41+
...
42+
# Subtest: top level skipped describe disabled
43+
ok 9 - top level skipped describe disabled # SKIP test name does not match pattern
44+
---
45+
duration_ms: *
46+
...
47+
# Subtest: top level skipped describe enabled
48+
ok 10 - top level skipped describe enabled # SKIP
49+
---
50+
duration_ms: *
51+
...
52+
# Subtest: top level runs because name includes PaTtErN
53+
ok 11 - top level runs because name includes PaTtErN
54+
---
55+
duration_ms: *
56+
...
57+
# Subtest: top level test enabled
58+
# Subtest: nested test runs because name includes PATTERN
59+
ok 1 - nested test runs because name includes PATTERN
60+
---
61+
duration_ms: *
62+
...
63+
1..1
64+
ok 12 - top level test enabled
65+
---
66+
duration_ms: *
67+
...
68+
# Subtest: top level describe enabled
69+
# Subtest: nested it disabled
70+
ok 1 - nested it disabled # SKIP test name does not match pattern
71+
---
72+
duration_ms: *
73+
...
74+
# Subtest: nested it enabled
75+
ok 2 - nested it enabled
76+
---
77+
duration_ms: *
78+
...
79+
# Subtest: nested describe disabled
80+
ok 3 - nested describe disabled # SKIP test name does not match pattern
81+
---
82+
duration_ms: *
83+
...
84+
# Subtest: nested describe enabled
85+
# Subtest: is enabled
86+
ok 1 - is enabled
87+
---
88+
duration_ms: *
89+
...
90+
1..1
91+
ok 4 - nested describe enabled
92+
---
93+
duration_ms: *
94+
...
95+
1..4
96+
ok 13 - top level describe enabled
97+
---
98+
duration_ms: *
99+
...
100+
1..13
101+
# tests 13
102+
# pass 4
103+
# fail 0
104+
# cancelled 0
105+
# skipped 9
106+
# todo 0
107+
# duration_ms *

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL