Repository navigation
Fix caching of PEP 561 namespace packages - #13124
Merged
JukkaL merged 3 commits intoJul 20, 2022
Merged
JukkaL merged 3 commits into
JukkaL merged 3 commits into
Conversation
Copy link
Copy Markdown
Collaborator
Sorry, something went wrong.
added 2 commits
July 14, 2022 18:18
This comment has been minimized.
Copy link
Copy Markdown
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: ignite (https://github.com/pytorch/ignite)
- ignite/contrib/engines/common.py:6: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ ignite/contrib/engines/common.py:10: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
poetry (https://github.com/python-poetry/poetry)
- src/poetry/config/config.py:13: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ src/poetry/config/config.py:14: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
arviz (https://github.com/arviz-devs/arviz)
+ arviz/plots/backends/matplotlib/traceplot.py:5: error: Cannot find implementation or library stub for module named "matplotlib"
- arviz/plots/backends/matplotlib/traceplot.py:6: error: Cannot find implementation or library stub for module named "matplotlib"
+ arviz/plots/backends/matplotlib/kdeplot.py:3: error: Cannot find implementation or library stub for module named "matplotlib"
- arviz/plots/backends/matplotlib/kdeplot.py:5: error: Cannot find implementation or library stub for module named "matplotlib"
+ arviz/plots/backends/matplotlib/elpdplot.py:4: error: Cannot find implementation or library stub for module named "matplotlib"
- arviz/plots/backends/matplotlib/elpdplot.py:5: error: Cannot find implementation or library stub for module named "matplotlib"
+ arviz/plots/backends/bokeh/khatplot.py:4: error: Cannot find implementation or library stub for module named "matplotlib"
- arviz/plots/backends/bokeh/khatplot.py:5: error: Cannot find implementation or library stub for module named "matplotlib"
vision (https://github.com/pytorch/vision)
- torchvision/__init__.py:4: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ torchvision/__init__.py:15: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
jax (https://github.com/google/jax)
- jax/_src/lib/__init__.py:38: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ jax/_src/lib/__init__.py:48: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
pyinstrument (https://github.com/joerick/pyinstrument)
+ pyinstrument/vendor/appdirs.py:533: error: Cannot find implementation or library stub for module named "com.sun"
- pyinstrument/vendor/appdirs.py:562: error: Cannot find implementation or library stub for module named "com.sun"
- pyinstrument/magic/magic.py:6: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ pyinstrument/magic/magic.py:7: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
bokeh (https://github.com/bokeh/bokeh)
- bokeh/core/serialization.py:49:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ bokeh/core/serialization.py:53:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
|
Sorry, something went wrong.
hauntsaninja
marked this pull request as ready for review
July 15, 2022 03:27
JukkaL
approved these changes
Jul 20, 2022
JukkaL
left a comment
Copy link
Copy Markdown
Collaborator
There was a problem hiding this comment.
Thanks for the fix! The logic at least looks more straightforward now, so hopefully it fixes the issue for good.
Sorry, something went wrong.
hauntsaninja
added a commit
to hauntsaninja/mypy
that referenced
this pull request
Apr 11, 2025
Fixes python#12664 A root cause is there is this stateful `_update_ns_ancestors` thing in `modulefinder`, so if things get called in the wrong order, you can get incorrect results. See also the logic in `all_imported_modules_in_file` where we've fixed several bugs like this previously, like python#13124 and python#10937 As a result of (seemingly accidentally) reusing imports across modules, we can end up in a situation where the namespace gets added as a dependency to other modules and so on the cached run we attempt to find namespace before package, which does not work I am not sure this `imports` code path is even needed, so I will open an alternate PR. I can't write a good test for this because it requires something in site_packages, but here's a minimal repro: ``` set -eux rm -rf repro mkdir repro cd repro SITEPACK=env/site-packages mkdir -p $SITEPACK mkdir $SITEPACK/ruamel mkdir $SITEPACK/ruamel/yaml printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py printf '' > $SITEPACK/ruamel/yaml/py.typed printf 'import ruamel.yaml' > a.py printf 'import a' > main.py rm -rf .mypy_cache PYTHONPATH=$SITEPACK mypy main.py PYTHONPATH=$SITEPACK mypy main.py ```
hauntsaninja
added a commit
that referenced
this pull request
Apr 11, 2025
Fixes #12664 A root cause is there is this stateful `_update_ns_ancestors` thing in `modulefinder`, so if things get called in the wrong order, you can get incorrect results. See also the logic in `all_imported_modules_in_file` where we've fixed several bugs like this previously, like #13124 and #10937 As a result of (seemingly accidentally) reusing imports across modules, we can end up in a situation where the namespace gets added as a dependency to all other modules and so on the cached run we attempt to find namespace before package, which does not work I am not sure this `imports` code path is even needed, so I will open an alternate PR, see #18908. Relevant history: - #6582 - #6179 I can't write a good test for this because it requires something in site_packages, but here's a minimal repro: ``` set -eux rm -rf repro mkdir repro cd repro SITEPACK=env/site-packages mkdir -p $SITEPACK mkdir $SITEPACK/ruamel mkdir $SITEPACK/ruamel/yaml printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py printf '' > $SITEPACK/ruamel/yaml/py.typed printf 'import ruamel.yaml' > a.py printf 'import a' > main.py rm -rf .mypy_cache PYTHONPATH=$SITEPACK mypy main.py PYTHONPATH=$SITEPACK mypy main.py ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #13085. Hopefully more robust than previous fixes along these lines.