gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') - #108480
Merged
serhiy-storchaka merged 4 commits intoSep 5, 2023
Merged
Conversation
…es_resource('walltime')
Copy link
Copy Markdown
Member
Author
|
These are mostly tests which contain several sleep()s or call a function which expected to fail by timeout. There are few tests which depend on slow external network service. |
Sorry, something went wrong.
gpshead
approved these changes
Sep 3, 2023
Copy link
Copy Markdown
Member
Author
$ ./python -m test -uall,-cpu -j4 ... Total duration: 7 min 27 sec Total tests: run=42,127 skipped=1,077 $ ./python -m test -uall,-cpu,-walltime -j4 ... Total duration: 6 min 26 sec Total tests: run=42,127 skipped=1,139 |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
|
Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.11 due to a conflict. |
Sorry, something went wrong.
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Sep 5, 2023
…es_resource('walltime') (pythonGH-108480)
(cherry picked from commit 1e0d627)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Copy link
Copy Markdown
|
GH-108923 is a backport of this pull request to the 3.12 branch. |
Sorry, something went wrong.
serhiy-storchaka
added a commit
to serhiy-storchaka/cpython
that referenced
this pull request
Sep 5, 2023
… requires_resource('walltime') (pythonGH-108480).
(cherry picked from commit 1e0d627)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Copy link
Copy Markdown
|
GH-108924 is a backport of this pull request to the 3.11 branch. |
Sorry, something went wrong.
serhiy-storchaka
added a commit
that referenced
this pull request
Sep 5, 2023
vstinner
pushed a commit
to vstinner/cpython
that referenced
this pull request
Sep 5, 2023
…es_resource('walltime') (pythonGH-108480)
(cherry picked from commit 1e0d627)
Yhg1s
pushed a commit
that referenced
this pull request
Sep 8, 2023
* gh-108834: regrtest reruns failed tests in subprocesses (#108839) When using --rerun option, regrtest now re-runs failed tests in verbose mode in fresh worker processes to have more deterministic behavior. So it can write its final report even if a test killed a worker progress. Add --fail-rerun option to regrtest: exit with non-zero exit code if a test failed pass passed when re-run in verbose mode (in a fresh process). That's now more useful since tests can pass when re-run in a fresh worker progress, whereas they failed when run after other tests when tests are run sequentially. Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a deprecated alias. Changes: * Fix and enhance statistics in regrtest summary. Add "(filtered)" when --match and/or --ignore options are used. * Add RunTests class. * Add TestResult.get_rerun_match_tests() method * Rewrite code to serialize/deserialize worker arguments as JSON using a new WorkerJob class. * Fix stats when a test is run with --forever --rerun. * If failed test names cannot be parsed, log a warning and don't filter tests. * test_regrtest.test_rerun_success() now uses a marker file, since the test is re-run in a separated process. * Add tests on normalize_test_name() function. * Add test_success() and test_skip() tests to test_regrtest. (cherry picked from commit 31c2945) * gh-108834: regrtest --fail-rerun exits with code 5 (#108896) When the --fail-rerun option is used and a test fails and then pass, regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test"). (cherry picked from commit 1170d5a) * gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480) (cherry picked from commit 1e0d627) * Manually sync Lib/test/libregrtest/ from main --------- Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.