FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

test: fix flaky wpt/test-timers by Trott · Pull Request #37691 · nodejs/node · GitHub

/ node Public

test: fix flaky wpt/test-timers - #37691

Merged
Trott merged 1 commit into
nodejs:masterfrom
Trott:fix-wpt-timers
Mar 14, 2021
Merged

test: fix flaky wpt/test-timers#37691
Trott merged 1 commit into
nodejs:masterfrom
Trott:fix-wpt-timers

Conversation

Trott commented Mar 10, 2021

Copy link
Copy Markdown
Member

Change the wpt tests to not run in parallel in our Python test runner.
On my local machine, this changes the time needed to run the suite from
about 2 seconds to about 5 seconds, but it makes the test suite much
more reliable.

Fixes: #37672

Trott added timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout. flaky-test Issues and PRs related to the tests with unstable failures on the CI. labels Mar 10, 2021
Trott requested a review from targos March 10, 2021 14:23
nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Mar 10, 2021

This comment has been minimized.

This comment has been minimized.

Copy link
Copy Markdown
Member

I don't understand this well enough to review but just wanted to say thanks for all the work on these tasks 🙇

Copy link
Copy Markdown
Collaborator

Copy link
Copy Markdown
Collaborator

This comment has been minimized.

This comment has been minimized.

Trott marked this pull request as draft March 11, 2021 06:14

targos commented Mar 11, 2021

Copy link
Copy Markdown
Member

Feel free to skip the tests again in the mean time

Trott commented Mar 11, 2021

Copy link
Copy Markdown
Member Author

The test that fails (negative-settimeout.any.js) is short and straightforward. It checks that a timer with a negative timeout value runs before a timer with a timeout value of 10.

setup({ single_test: true });
setTimeout(done, -100);
setTimeout(assert_unreached, 10);

While the ordering may be guaranteed in browsers, I don't think it is in Node.js due to the linked list stuff that underpins timers. But maybe I'm confused. /ping @nodejs/timers for confirmation.

If the ordering in the above case is not in fact guaranteed in Node.js, then skipping the test is certainly the right thing to do.

Linkgoron commented Mar 11, 2021
edited
Loading

Copy link
Copy Markdown
Contributor

The test that fails (negative-settimeout.any.js) is short and straightforward. It checks that a timer with a negative timeout value runs before a timer with a timeout value of 10.

setup({ single_test: true });
setTimeout(done, -100);
setTimeout(assert_unreached, 10);

While the ordering may be guaranteed in browsers, I don't think it is in Node.js due to the linked list stuff that underpins timers. But maybe I'm confused. /ping @nodejs/timers for confirmation.

If the ordering in the above case is not in fact guaranteed in Node.js, then skipping the test is certainly the right thing to do.

@Trott I think that there's something wrong with done(). Rewriting the test to the following is not flaky on my machine (while the original is):

setup({ single_test: true });
setTimeout(() => { clearTimeout(y); done(); }, -100);
var y = setTimeout(assert_unreached, 10);

I assume that done doesn't clear the timeout, and there's an edge-case where if there's a very small difference (or maybe if they're on the same tick) between the actual running time of done and assert_unreached it fails the test anyway. I think that the above fix is a slightly more correct way to test that the first runs before the second anyway.

Trott marked this pull request as ready for review March 13, 2021 16:32

Trott commented Mar 13, 2021

Copy link
Copy Markdown
Member Author

OK. The Parallel to Simple change fixed most of them. The one it didn't fix is now marked flaky in the status file. This is ready for review.

nodejs-github-bot commented Mar 13, 2021
edited by Trott
Loading

Copy link
Copy Markdown
Collaborator

Trott commented Mar 13, 2021
edited
Loading

Copy link
Copy Markdown
Member Author

Stress test against this PR: https://ci.nodejs.org/view/Stress/job/node-stress-single-test/224/

Trott commented Mar 13, 2021

Copy link
Copy Markdown
Member Author

Stress test and CI were both green on the first run. It would be great to get this frequent-CI-failure addressed. @nodejs/testing

targos left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks!

Moving test harness from Parallel to Simple resolves most of the
failures. negative-settimeout.any.js still needs to be marked flaky.

Refs: nodejs#37672

PR-URL: nodejs#37691
Fixes: nodejs#37672
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>

Trott commented Mar 14, 2021

Copy link
Copy Markdown
Member Author

Landed in 66f0eb7"

Trott merged commit 66f0eb7 into nodejs:master Mar 14, 2021
Trott deleted the fix-wpt-timers branch March 14, 2021 18:38
danielleadams pushed a commit that referenced this pull request Mar 16, 2021
Moving test harness from Parallel to Simple resolves most of the
failures. negative-settimeout.any.js still needs to be marked flaky.

Refs: #37672

PR-URL: #37691
Fixes: #37672
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
danielleadams pushed a commit that referenced this pull request Mar 16, 2021
Moving test harness from Parallel to Simple resolves most of the
failures. negative-settimeout.any.js still needs to be marked flaky.

Refs: #37672

PR-URL: #37691
Fixes: #37672
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky-test Issues and PRs related to the tests with unstable failures on the CI. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. timers Issues and PRs related to the timers subsystem / setImmediate, setInterval, setTimeout.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

investigate flaky wpt/test-timers

6 participants


Back | FazBrowse Home | New Git URL