FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-62432: Return unsuccessfully if no unit tests were run by mgorny · Pull Request #24893 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-62432: Return unsuccessfully if no unit tests were run #24893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mgorny wants to merge 1 commit into from
Conversation 5 Commits 1 Checks 0 Files changed
Closed

gh-62432: Return unsuccessfully if no unit tests were run #24893

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension .py  (3) .rst  (2) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
7 changes: 5 additions & 2 deletions Doc/library/unittest.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -2012,13 +2012,16 @@ Loading and running tests

.. method:: wasSuccessful()

Return ``True`` if all tests run so far have passed, otherwise returns
``False``.
Return ``True`` if all tests run so far have passed and at least one test
was found, otherwise returns ``False``.

iritkatriel May 7, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

The code checks testsRun + testsSkipped > 0, so it's not exactly "was found" (what if many were found but we haven't started running anything yet?)

nit: I think it's easier to understand if the sentence lists the conditions chronologically, something like: "at least one test has been executed or skipped and no test has failed so far".

.. versionchanged:: 3.4
Returns ``False`` if there were any :attr:`unexpectedSuccesses`
from tests marked with the :func:`expectedFailure` decorator.

.. versionchanged:: 3.10b1

iritkatriel May 7, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This will probably need to be 3.11 now.
Returns ``False`` if no tests were found.

.. method:: stop()

This method can be called to signal that the set of tests being run should
Expand Down
7 changes: 6 additions & 1 deletion Lib/unittest/result.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -159,10 +159,15 @@ def addUnexpectedSuccess(self, test):

def wasSuccessful(self):
"""Tells whether or not this result was a success."""
# testsRun > 0 ensures that we do not return success if test
# discovery failed. We additionally need to check skipped list
# since class/module-level skips do not count towards testsRun.
#
# The hasattr check is for test_result's OldResult test. That
# way this method works on objects that lack the attribute.
# (where would such result instances come from? old stored pickles?)
return ((len(self.failures) == len(self.errors) == 0) and
return ((self.testsRun > 0 or len(getattr(self, 'skipped', ())) > 0) and
(len(self.failures) == len(self.errors) == 0) and
(not hasattr(self, 'unexpectedSuccesses') or
len(self.unexpectedSuccesses) == 0))

Expand Down
7 changes: 5 additions & 2 deletions Lib/unittest/runner.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,11 @@ def run(self, test):
if hasattr(result, 'separator2'):
self.stream.writeln(result.separator2)
run = result.testsRun
self.stream.writeln("Ran %d test%s in %.3fs" %
(run, run != 1 and "s" or "", timeTaken))
if run > 0 or len(result.skipped) > 0:

iritkatriel May 7, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'd abstract the run > 0 or len(result.skipped) calculation as a method on TestResult, then you won't have it repeated twice. And you can give it a name indicating what it means.
self.stream.writeln("Ran %d test%s in %.3fs" %
(run, run != 1 and "s" or "", timeTaken))
else:
self.stream.writeln("No tests found")
self.stream.writeln()

expectedFails = unexpectedSuccesses = skipped = 0
Expand Down
52 changes: 51 additions & 1 deletion Lib/unittest/test/test_result.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import traceback
import unittest
from unittest.suite import _ErrorHolder


class MockTraceback(object):
Expand Down Expand Up @@ -35,9 +36,10 @@ class Test_TestResult(unittest.TestCase):
def test_init(self):
result = unittest.TestResult()

self.assertTrue(result.wasSuccessful())
self.assertFalse(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 0)
self.assertEqual(result.shouldStop, False)
self.assertIsNone(result._stdout_buffer)
Expand Down Expand Up @@ -69,6 +71,7 @@ def test_1(self):
self.assertTrue(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand All @@ -90,6 +93,7 @@ def test_1(self):
self.assertTrue(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand Down Expand Up @@ -143,6 +147,7 @@ def test_1(self):
self.assertTrue(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand Down Expand Up @@ -186,6 +191,7 @@ def test_1(self):
self.assertFalse(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 1)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand Down Expand Up @@ -234,6 +240,7 @@ def test_1(self):
self.assertFalse(result.wasSuccessful())
self.assertEqual(len(result.errors), 1)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand All @@ -260,6 +267,48 @@ def test_1(self):
test_case, formatted_exc = result.errors[0]
self.assertEqual('A tracebacklocals', formatted_exc)

def test_addSkip(self):
class Foo(unittest.TestCase):
def test_1(self):
pass

test = Foo('test_1')

result = unittest.TestResult()

result.startTest(test)
result.addSkip(test, "skipped")
result.stopTest(test)

self.assertTrue(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 1)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

test_case, reason = result.skipped[0]
self.assertIs(test_case, test)
self.assertEqual(reason, "skipped")

def test_addSkipClassLevel(self):
"""Test for SkipTest happening at the class level"""
error = _ErrorHolder('setUpClass (test_foo.Foo)')

result = unittest.TestResult()
result.addSkip(error, "skipped")

self.assertTrue(result.wasSuccessful())
self.assertEqual(len(result.errors), 0)
self.assertEqual(len(result.failures), 0)
self.assertEqual(len(result.skipped), 1)
self.assertEqual(result.testsRun, 0)
self.assertEqual(result.shouldStop, False)

test_case, reason = result.skipped[0]
self.assertIs(test_case, error)
self.assertEqual(reason, "skipped")

def test_addSubTest(self):
class Foo(unittest.TestCase):
def test_1(self):
Expand All @@ -284,6 +333,7 @@ def test_1(self):
self.assertFalse(result.wasSuccessful())
self.assertEqual(len(result.errors), 1)
self.assertEqual(len(result.failures), 1)
self.assertEqual(len(result.skipped), 0)
self.assertEqual(result.testsRun, 1)
self.assertEqual(result.shouldStop, False)

Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2021-03-16-13-15-44.bpo-18232.0CSR5D.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
:func:`unittest.TestResult.wasSuccessful` now returns False if no tests were
found.
Back | FazBrowse Home | New Git URL