gh-106584: Fix exit code for unittest in Python 3.12 by EliseevEgor · Pull Request #106588 · python/cpython · GitHub

Increase the number of running tests only if a test hasn't been skipped.

Issue: Return the correct exit code if all tests have been skipped in unittest #106584

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

All commit authors signed the Contributor License Agreement.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Increase the number of running tests only if it hasn't been skipped.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

LGTM, but please add a NEWS entry!

…-SBtC.rst

🤖 New build scheduled with the buildbot fleet by @sunmy2019 for commit 9779a2b 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

No need to specify the version info in the NEWS.

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

🤖 New build scheduled with the buildbot fleet by @sunmy2019 for commit 6313c98 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

We got some failed tests. Most are unrelated, but we might need to rule them out.

…ss_db_feature() test on Python 3.12.1+. Python 3.12.1+ no longer includes skipped tests in the number of running tests. Check out: python/cpython#110890 (comment) python/cpython#106588

…ass_unless_db_feature() test on Python 3.12.1+. Python 3.12.1+ no longer includes skipped tests in the number of running tests. Check out: python/cpython#110890 (comment) python/cpython#106588 Backport of 20b7aac from main

It doesn't seem like this ought to have been backported. It includes an incompatible change in the long-established protocol between a test runner and a test results object. One consequence is that it has broken trial's multiprocess test runner (twisted/twisted#12052).

In Python 3.12.1 python/cpython#106588 was backported which changed the execution behavior of the unittest runner. After python/cpython#106588 startTest() is no longer being called if a test is skipped. This causes knock-on effects in testtools because the test result subclasses were assuming that startTest() was always called when stopTest() was called. To handle this change in behavior when running with Python 3.12.1 this commit adds a check to only deal with tags if they exist (when startTest() is run).

FYI-- This also broke absl.testing where it tracks test runtime with startTest and stopTest: https://github.com/abseil/abseil-py/blob/2af184bfcdaa6156a5a098923a9561bc4b7d988c/absl/testing/xml_reporter.py#L352-L365

We should be able to workaround it, but just wanted to point this out. It seems strange for stopTest to be called without a startTest.

…ipped. `startTest` may not be called in this case, after the change in python/cpython#106588. PiperOrigin-RevId: 597675639

…ipped. `startTest` may not be called in this case, after the change in python/cpython#106588. PiperOrigin-RevId: 597840713

In Python 3.12.1 python/cpython#106588 was backported which changed the execution behavior of the unittest runner. After python/cpython#106588 startTest() is no longer being called if a test is skipped. This causes knock-on effects in testtools because the test result subclasses were assuming that startTest() was always called when stopTest() was called. To handle this change in behavior when running with Python 3.12.1 this commit adds a check to only deal with tags if they exist (when startTest() is run).

* Add Python 3.12 builds The migrator seems confused by another package from a split feedstock that also produces non-Python packages that qiskit-aer depends on not having Python 3.12 builds yet, so this is a manual migration. Explicitly exclude TestNoiseTransformer tests because they require cvxpy. Previously they were skipped by unittest when cvxpy was not installed but test skipping was broken by Python 3.12.1 (see testing-cabal/testtools#371 and python/cpython#106588), so we had to explicitly skip the tests with the test runner.

…ython#106588)" This reverts commit 8fc0713.

…12 (#106588)" (GH-114470) This reverts commit 8fc0713.

…in Python 3.12 (pythonGH-106588)" (pythonGH-114470) This reverts commit 8fc0713. (cherry picked from commit ecabff9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…thon 3.12 (GH-106588)" (GH-114470) (GH-114994) This reverts commit 8fc0713. (cherry picked from commit ecabff9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

…6588) Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-bot added the awaiting review label Jul 10, 2023

bedevere-bot mentioned this pull request Jul 10, 2023

Return the correct exit code if all tests have been skipped in unittest #106584

Closed

EliseevEgor force-pushed the egoreliseev/Issue-106584_move_start_test branch from 2b0b119 to 9efdf58 Compare July 10, 2023 12:48

pythonGH-106584 : Fix exit code for unittest in Python 3.12

6e1be71

Increase the number of running tests only if it hasn't been skipped.

EliseevEgor force-pushed the egoreliseev/Issue-106584_move_start_test branch from 9efdf58 to 6e1be71 Compare July 10, 2023 12:51

sobolevn approved these changes Jul 11, 2023

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Jul 11, 2023

📜🤖 Added by blurb_it.

ee3af81

sunmy2019 reviewed Jul 11, 2023

View reviewed changes

Comment thread Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst Outdated

Update Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g…

9779a2b

…-SBtC.rst

sunmy2019 reviewed Jul 11, 2023

View reviewed changes

Comment thread Lib/unittest/case.py Outdated

sunmy2019 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

pythonGH-106584 : Apply comment suggestion

43248a9

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

sobolevn reviewed Jul 11, 2023

View reviewed changes

Comment thread Misc/NEWS.d/next/Library/2023-07-11-08-56-40.gh-issue-106584.g-SBtC.rst Outdated

EliseevEgor and others added 3 commits July 11, 2023 14:04

pythonGH-106584 : Remove the version info in the NEWS

901cf01

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

pythonGH-106584 : Remove empty line in the NEWS

1ab5cfa

pythonGH-106584 : Check buffers before seek

6313c98

sunmy2019 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2023

Merge branch 'main' into egoreliseev/Issue-106584_move_start_test

046ae78

terryjreedy mentioned this pull request Oct 15, 2023

unittest.TestResult.testsRun no longer counts skipped tests. #110890

Closed

felixxm mentioned this pull request Oct 15, 2023

Refs #34900, Refs #34118 -- Updated assertion in test_skip_class_unless_db_feature() test on Python 3.12.1+. django/django#17367

Merged

mtreinish mentioned this pull request Jan 5, 2024

Handle breaking api change in Python 3.12.1 testing-cabal/testtools#371

Merged

copybara-service Bot mentioned this pull request Jan 11, 2024

Fix absl.testing.xml_reporter for Python 3.12.1 when all tests are skipped. abseil/abseil-py#271

Merged

wshanks mentioned this pull request Jan 19, 2024

Add Python 3.12 builds conda-forge/qiskit-aer-feedstock#31

Merged

4 of 5 tasks

serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Jan 23, 2024

Revert "pythongh-106584 : Fix exit code for unittest in Python 3.12 (p…

15d1968

…ython#106588)" This reverts commit 8fc0713.

serhiy-storchaka added a commit that referenced this pull request Feb 4, 2024

gh-113267 : Revert "gh-106584 : Fix exit code for unittest in Python 3.…

ecabff9

…12 (#106588)" (GH-114470) This reverts commit 8fc0713.

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-113267 : Revert "pythongh-106584 : Fix exit code for unittest …

9d44784

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

fsc-eriker pushed a commit to fsc-eriker/cpython that referenced this pull request Feb 14, 2024

pythongh-113267 : Revert "pythongh-106584 : Fix exit code for unittest …

39378cc

…in Python 3.12 (python#106588)" (pythonGH-114470) This reverts commit 8fc0713.

gh-106584: Fix exit code for unittest in Python 3.12 by EliseevEgor · Pull Request #106588 · python/cpython · GitHub

Repository navigation

gh-106584: Fix exit code for unittest in Python 3.12 - #106588

gh-106584: Fix exit code for unittest in Python 3.12#106588ambv merged 8 commits intopython:mainpython/cpython:mainfrom EliseevEgor:egoreliseev/Issue-106584_move_start_testEliseevEgor/cpython:egoreliseev/Issue-106584_move_start_testCopy head branch name to clipboard

Conversation

EliseevEgor commented Jul 10, 2023 • edited by terryjreedy Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Jul 10, 2023

Uh oh!

ghost commented Jul 10, 2023 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Jul 10, 2023

Uh oh!

bedevere-bot commented Jul 10, 2023

Uh oh!

bedevere-bot commented Jul 10, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Jul 11, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-bot commented Jul 11, 2023

Uh oh!

sunmy2019 commented Jul 13, 2023

Uh oh!

exarkun commented Dec 18, 2023

Uh oh!

yilei commented Jan 11, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

gh-106584: Fix exit code for unittest in Python 3.12#106588
ambv merged 8 commits into
python:mainfrom
EliseevEgor:egoreliseev/Issue-106584_move_start_test

EliseevEgor commented Jul 10, 2023 •
edited by terryjreedy
Loading

ghost commented Jul 10, 2023 •
edited by ghost
Loading