gh-111681: Fix doctests in `typing.rst` and remove unused imports by sobolevn · Pull Request #111682 · python/cpython · GitHub

Without globs it would fail:

» ./python.exe -m test test_typing
Using random seed: 1924694082
0:00:00 load avg: 1.95 Run 1 test sequentially
0:00:00 load avg: 1.95 [1/1] test_typing
test test_typing failed -- Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython2/Lib/doctest.py", line 2263, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for typing.rst
  File "/Users/sobolev/Desktop/cpython2/Doc/library/typing.rst", line 0

----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython2/Doc/library/typing.rst", line 648, in typing.rst
Failed example:
    Z[int, [dict, float]]
Expected:
    __main__.Z[int, [dict, float]]
Got:
    Z[int, [dict, float]]
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython2/Doc/library/typing.rst", line 669, in typing.rst
Failed example:
    X[int, str]
Expected:
    __main__.X[[int, str]]
Got:
    X[[int, str]]
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython2/Doc/library/typing.rst", line 671, in typing.rst
Failed example:
    X[[int, str]]
Expected:
    __main__.X[[int, str]]
Got:
    X[[int, str]]
----------------------------------------------------------------------
File "/Users/sobolev/Desktop/cpython2/Doc/library/typing.rst", line 1996, in typing.rst
Failed example:
    Alias.__module__
Expected:
    '__main__'
Got nothing

Issue: typing.rst contains failing doctests #111681

📚 Documentation preview 📚: https://cpython-previews--111682.org.readthedocs.build/

I think we already run these doctests via Sphinx in CI. If there are certain doctests that aren't being run via that CI job, then I think we should investigate why that is, rather than running the entire doctest suite twice in CI

I think we already run these doctests via Sphinx in CI.

See https://github.com/python/cpython/actions/runs/6741804289/job/18326927659 for a recent run, in which 55/55 typing doctests were reported to have passed (but maybe there's some that the job missed)

@AlexWaygood as you can see there's something wrong with Doctest CI job: https://github.com/python/cpython/actions/runs/6743114958/job/18330437061?pr=111682 It passes on a clearly broken example. But, my new job fails as it should: https://github.com/python/cpython/actions/runs/6743114958/job/18330434893?pr=111682

I know this because I've already fixed multiple doctests for other modules.

@AlexWaygood as you can see there's something wrong with Doctest CI job: https://github.com/python/cpython/actions/runs/6743114958/job/18330437061?pr=111682 It passes on a clearly broken example.

In which case, I think we should either try to fix the broken CI job, or, if it's useless, remove it from CI. @AA-Turner, any idea why the doctest CI job doesn't seem to be doing what it should be here? :)

this isn't actually doctested, I don't think, since you don't have the .. doctest:: directive above this block. (I've just pushed to your branch to add it.)

It is a doctest, clearly:

» ./python.exe -m doctest Doc/library/typing.rst
**********************************************************************
File "Doc/library/typing.rst", line 651, in typing.rst
Failed example:
    1 + 2
Expected:
    4
Got:
    3
**********************************************************************
1 items had failures:
   1 of  48 in typing.rst
***Test Failed*** 1 failures.

It works the same way with and without .. doctest::.
The problem might that Sphinx does not recognize doctests without this directive, but this is clearly a Sphinx's own issue.

Yes, that was what I meant, sorry for not being clear: that it wouldn't be picked up by sphinx's doctest job :)

We might need to use doctest_test_doctest_blocks from https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

We might need to use doctest_test_doctest_blocks from https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

We can try enabling that, sure, but that should be a separate issue/PR. This PR is about fixing broken doctests in typing.rst, not changing the way we determine in CI which doctests should be run. The majority of these that need to be tested already are tested, using the .. doctest:: directive.

It seems like the current CI job is not broken at all. The "clearly broken" example you added correctly failed in CI with the Sphinx doctest job after the .. doctest:: marker was added: https://github.com/python/cpython/actions/runs/6743594059/job/18331803827?pr=111682

I don't mind the change you're making here. But the reason why this doesn't currently fail when Sphinx runs our doctests in CI is because of this, which is run before every doctest block:

cpython/Doc/library/typing.rst

Lines 5 to 9 in d49aba5

    
           .. testsetup:: * 
        
              import typing 
        
              from dataclasses import dataclass 
        
              from typing import *

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

It seems like the current CI job is not broken at all

It might not be broken, it just allows a lot of broken doctests in.
I think that regular doctests must always work, not just some sub-selection of them.

Because what happens in reality is that a lot of code blocks / example do not have .. doctest directive and they become outdated, maintainers do not notice it, users get upset.

Because what happens in reality is that a lot of code blocks / example do not have .. doctest directive and they become outdated, maintainers do not notice it, users get upset.

I agree this situation is suboptimal. But that makes me think that we should do one of the following two things:

Get rid of the current doctest job in CI, try to make sure all doctests in all the docs are run via unittest
Explore using doctest_test_doctest_blocks, as you suggested in gh-111681: Fix doctests in typing.rst and remove unused imports #111682 (comment), so that all doctest blocks in all the docs are tested by Sphinx by default.

It makes no sense to me to do what this PR currently proposes: run most of the doctests in typing.rst twice in CI.

Yes, I agree. I will convert it to a very minimalistic one: just remove unused imports and fix doctests a bit, without adding new tests to the suite.

Thanks!

Thanks @sobolevn for the PR, and @AlexWaygood for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

Sorry, @sobolevn and @AlexWaygood, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ccc8caa8587103c4ccf617ba106cef63344504dd 3.12

Sorry, @sobolevn and @AlexWaygood, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ccc8caa8587103c4ccf617ba106cef63344504dd 3.11

@sobolevn, can I leave the backports with you? :)

Will do, I will also create a new issue about our CI + doctest.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows11 Bigmem 3.x has failed when building commit ccc8caa.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1079/builds/2658) and take a look at the build logs.
Check if the failure is related to this commit (ccc8caa) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1079/builds/2658

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 8, done.        
remote: Counting objects:  12% (1/8)        
remote: Counting objects:  25% (2/8)        
remote: Counting objects:  37% (3/8)        
remote: Counting objects:  50% (4/8)        
remote: Counting objects:  62% (5/8)        
remote: Counting objects:  75% (6/8)        
remote: Counting objects:  87% (7/8)        
remote: Counting objects: 100% (8/8)        
remote: Counting objects: 100% (8/8), done.        
remote: Compressing objects:  12% (1/8)        
remote: Compressing objects:  25% (2/8)        
remote: Compressing objects:  37% (3/8)        
remote: Compressing objects:  50% (4/8)        
remote: Compressing objects:  62% (5/8)        
remote: Compressing objects:  75% (6/8)        
remote: Compressing objects:  87% (7/8)        
remote: Compressing objects: 100% (8/8)        
remote: Compressing objects: 100% (8/8), done.        
remote: Total 8 (delta 0), reused 1 (delta 0), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'ccc8caa8587103c4ccf617ba106cef63344504dd'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at ccc8caa858 gh-111681: minor fixes to typing doctests; remove unused imports in `test_typing` (#111682)
Switched to and reset branch 'main'

Could Not Find R:\buildarea\3.x.ambv-bb-win11.bigmem\build\Lib\*.pyc
The system cannot find the file specified.
Could Not Find R:\buildarea\3.x.ambv-bb-win11.bigmem\build\PCbuild\python*.zip

Could Not Find R:\buildarea\3.x.ambv-bb-win11.bigmem\build\PCbuild\python*.zip

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.x has failed when building commit ccc8caa.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/366/builds/5809) and take a look at the build logs.
Check if the failure is related to this commit (ccc8caa) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/366/builds/5809

Summary of the results of the build (if available):

==

Click to see traceback logs

Note: switching to 'ccc8caa8587103c4ccf617ba106cef63344504dd'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at ccc8caa858 gh-111681: minor fixes to typing doctests; remove unused imports in `test_typing` (#111682)
Switched to and reset branch 'main'

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.13d.a(dynamic_annotations.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.13d.a(pymath.o) has no symbols
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups

make: *** [buildbottest] Error 3

…s in `test_typing` (python#111682) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

GH-112035 is a backport of this pull request to the 3.12 branch.

GH-112037 is a backport of this pull request to the 3.11 branch.

…ts in `test_typing` (#111682) (#112035) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…s in `test_typing` (python#111682) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

pythongh-111681 : Fix doctests in typing.rst and run it in CI

397f51b

sobolevn requested review from AlexWaygood, JelleZijlstra and gvanrossum as code owners November 3, 2023 08:18

bedevere-app Bot added the awaiting review label Nov 3, 2023

bedevere-app Bot mentioned this pull request Nov 3, 2023

typing.rst contains failing doctests #111681

Closed

bedevere-app Bot added the tests Tests in the Lib/test dir label Nov 3, 2023

sobolevn added the skip news label Nov 3, 2023

AlexWaygood reviewed Nov 3, 2023

View reviewed changes

Comment thread Doc/library/typing.rst Outdated

Test if Sphinx can catch obvious errors

cd22a87

AlexWaygood reviewed Nov 3, 2023

View reviewed changes

Update typing.rst

a4c7b5f

AlexWaygood requested changes Nov 3, 2023

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting review labels Nov 3, 2023

Only keep small changes

9461271

sobolevn changed the title gh-111681: Fix doctests in typing.rst and run it in CI gh-111681: Fix doctests in typing.rst and remove unused imports Nov 3, 2023

sobolevn and others added 2 commits November 3, 2023 14:29

Typo

d7eae8b

add .. doctest:: to the failing one to prevent regressions

c84f300

AlexWaygood approved these changes Nov 3, 2023

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting changes labels Nov 3, 2023

AlexWaygood added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Nov 3, 2023

AlexWaygood enabled auto-merge (squash) November 3, 2023 11:49

AlexWaygood merged commit ccc8caa into python:main Nov 3, 2023

bedevere-app Bot removed the awaiting merge label Nov 3, 2023

miss-islington-app Bot assigned AlexWaygood Nov 3, 2023

AlexWaygood assigned sobolevn and unassigned AlexWaygood Nov 3, 2023

sobolevn mentioned this pull request Nov 3, 2023

Decide what should we do with doctests in CI #111704

Open

AlexWaygood added a commit to AlexWaygood/cpython that referenced this pull request Nov 9, 2023

pythongh-111681 : minor fixes to typing doctests; remove unused import…

81e23e7

…s in `test_typing` (python#111682) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

bedevere-app Bot removed the needs backport to 3.12 only security fixes label Nov 13, 2023

bedevere-app Bot removed the needs backport to 3.11 only security fixes label Nov 13, 2023

AlexWaygood added a commit to AlexWaygood/cpython that referenced this pull request Nov 13, 2023

[3.11] pythongh-111681 : minor fix to a typing doctest (python#111682 )

904084a

AlexWaygood added a commit that referenced this pull request Nov 13, 2023

[3.11] gh-111681 : minor fix to a typing doctest (#111682 ) (#112037 )

4baf633

sobolevn mentioned this pull request Nov 16, 2023

typing.py contains two doctests that are not executed, ever #112155

Closed

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-111681 : minor fixes to typing doctests; remove unused import…

f324431

…s in `test_typing` (python#111682) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-111681 : minor fixes to typing doctests; remove unused import…

6e909e1

…s in `test_typing` (python#111682) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

                  .. doctest::
-                    >>> from typing import ParamSpec
+                    >>> from typing import ParamSpec, get_origin

	.. testsetup:: *

	import typing
	from dataclasses import dataclass
	from typing import *

gh-111681: Fix doctests in `typing.rst` and remove unused imports by sobolevn · Pull Request #111682 · python/cpython · GitHub

Repository navigation

gh-111681: Fix doctests in typing.rst and remove unused imports - #111682

gh-111681: Fix doctests in typing.rst and remove unused imports#111682AlexWaygood merged 6 commits intopython:mainpython/cpython:mainfrom sobolevn:issue-111681sobolevn/cpython:issue-111681Copy head branch name to clipboard

Conversation

sobolevn commented Nov 3, 2023 • edited by hugovk Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

AlexWaygood commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Nov 3, 2023

Uh oh!

AlexWaygood commented Nov 3, 2023

Uh oh!

AlexWaygood Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sobolevn Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

sobolevn Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

AlexWaygood Nov 3, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-app Bot commented Nov 3, 2023

Uh oh!

sobolevn commented Nov 3, 2023

Uh oh!

AlexWaygood commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexWaygood left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington-app Bot commented Nov 3, 2023

Uh oh!

miss-islington-app Bot commented Nov 3, 2023

Uh oh!

miss-islington-app Bot commented Nov 3, 2023

Uh oh!

AlexWaygood commented Nov 3, 2023

Uh oh!

sobolevn commented Nov 3, 2023

Uh oh!

bedevere-bot commented Nov 3, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Nov 3, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-app Bot commented Nov 13, 2023

Uh oh!

bedevere-app Bot commented Nov 13, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

gh-111681: Fix doctests in typing.rst and remove unused imports#111682
AlexWaygood merged 6 commits into
python:mainfrom
sobolevn:issue-111681

sobolevn commented Nov 3, 2023 •
edited by hugovk
Loading

AlexWaygood commented Nov 3, 2023 •
edited
Loading

AlexWaygood commented Nov 3, 2023 •
edited
Loading

AlexWaygood Nov 3, 2023 •
edited
Loading

sobolevn Nov 3, 2023 •
edited
Loading

AlexWaygood commented Nov 3, 2023 •
edited
Loading

sobolevn commented Nov 3, 2023 •
edited
Loading