Review requested:

• @nodejs/test_runner

CI: https://ci.nodejs.org/job/node-test-pull-request/49739/

@nodejs/test_runner this only solves one issue, there seems to be another issue with the process not being alive long enough to pipe the entire stream, see
https://ci.nodejs.org/job/node-test-commit-plinux/nodes=rhel8-ppc64le/47011/
and #22088 (comment)

not ok 9 - should support a 'js' file as a custom reporter
21:45:52           ---
21:45:52           duration_ms: 1125.36918
21:45:52           failureType: 'testCodeFailure'
21:45:52           error: |-
21:45:52             The input did not match the regular expression /{"test:start":4,"test:pass":2,"test:fail":2,"test:plan":2,"test:diagnostic":\d+}$/. Input:
21:45:52             
21:45:52             'custom.js {"test:start":1}'
21:45:52             
21:45:52           code: 'ERR_ASSERTION'
21:45:52           actual: 'custom.js {"test:start":1}'
21:45:52           operator: 'match'
21:45:52           stack: |-
21:45:52             Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-plinux/nodes/rhel8-ppc64le/test/parallel/test-runner-reporters.js:93:14)
21:45:52             ItTest.runInAsyncScope (node:async_hooks:206:9)
21:45:52             ItTest.run (node:internal/test_runner/test:552:25)
21:45:52             ItTest.start (node:internal/test_runner/test:468:17)
21:45:52             node:internal/test_runner/test:811:71
21:45:52             node:internal/per_context/primordials:470:82
21:45:52             new Promise (<anonymous>)
21:45:52             new SafePromise (node:internal/per_context/primordials:438:29)
21:45:52             node:internal/per_context/primordials:470:9
21:45:52             Array.map (<anonymous>)

I am trying to work on the additional issue on a separate PR

CI: https://ci.nodejs.org/job/node-test-pull-request/49746/

Should we wait for the child process's 'close' event instead of 'exit'?

Should we wait for the child process's 'close' event instead of 'exit'?

once the child process is not alive, transforming the output asynchronously is not enough for keeping the main process alive

I have solved that like this:

@@ -123,13 +125,16 @@ function setup(root) {
   const rejectionHandler =
     createProcessEventHandler('unhandledRejection', root);
   const coverage = configureCoverage(root);
-  const exitHandler = () => {
+  const exitHandler = async () => {
     root.coverage = collectCoverage(root, coverage);
     root.postRun(new ERR_TEST_FAILURE(
       'Promise resolution is still pending but the event loop has already resolved',
       kCancelledByParent));
 
     hook.disable();
+    const handle = setInterval(() => {}, 1000);
+    await finished(root.reporter);
+    clearInterval(handle);
     process.removeListener('unhandledRejection', rejectionHandler);
     process.removeListener('uncaughtException', exceptionHandler);
   };

but that is kind of a hack

if anyone wants to tackle this it reproduces running tools/test.py -J test/parallel/test-runner-reporters.js --repeat 1000

it reproduces running

Interestingly, I haven't been able to get it to reproduce on my machine (M1 mac).

but that is kind of a hack

Definitely a hack, but if there is no way to keep the process alive, I'd be OK with doing something like that, but I think it should exist in run() and the interval period should be large.

Interestingly, I haven't been able to get it to reproduce on my machine (M1 mac).

Mine is a M1 mac as well 😕

I think it should exist in run()

@cjihrig I have tried that but it creates a deadlock - adding a timer prevents the process from reaching beforeExit since the event loop hasn't ended,
but the reporter/s are not ended/drained yet since root.postRun is only called within beforeExit
WDYT?

CI: https://ci.nodejs.org/job/node-test-pull-request/49846/

CI: https://ci.nodejs.org/job/node-test-pull-request/49848/

- Loading data for nodejs/node/pull/46737
✔  Done loading data for nodejs/node/pull/46737
----------------------------------- PR info ------------------------------------
Title      test_runner: bootstrap reporters before running tests (#46737)
Author     Moshe Atlow  (@MoLow)
Branch     MoLow:wait-for-parser-to-finish -> nodejs:main
Labels     flaky-test, author ready, needs-ci, dont-land-on-v14.x, test_runner
Commits    1
 - test_runner: bootstrap reporters before running tests
Committers 1
 - Moshe Atlow 
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum 
Reviewed-By: Colin Ihrig 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum 
Reviewed-By: Colin Ihrig 
--------------------------------------------------------------------------------
   ℹ  This PR was created on Sun, 19 Feb 2023 19:16:17 GMT
   ✔  Approvals: 2
   ✔  - Benjamin Gruenbaum (@benjamingr): https://github.com/nodejs/node/pull/46737#pullrequestreview-1308076791
   ✔  - Colin Ihrig (@cjihrig) (TSC): https://github.com/nodejs/node/pull/46737#pullrequestreview-1307932394
   ✖  GitHub CI is still running
   ℹ  Last Full PR CI on 2023-02-21T19:30:15Z: https://ci.nodejs.org/job/node-test-pull-request/49848/
- Querying data for job/node-test-pull-request/49848/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

Landed in ce49f79