FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Fix incremental issue with namespace packages (option 2) by hauntsaninja · Pull Request #18908 · python/mypy · GitHub

Repository navigation

Fix incremental issue with namespace packages (option 2) - #18908

Closed
hauntsaninja wants to merge 1 commit into
python:masterfrom
hauntsaninja:ruamelfix2
Closed

hauntsaninja wants to merge 1 commit into
python:masterfrom
hauntsaninja:ruamelfix2

Conversation

hauntsaninja commented Apr 11, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Fixes #12664

See #18907 which is another possible fix

I'm not sure why this logic is actually needed given that we should record the dependency from import analysis in fastparse... maybe just because it adjusts priority?

Relevant history:

I can't write a good test for this because it requires something in site_packages, but here's a minimal repro:

set -eux
rm -rf repro
mkdir repro
cd repro

SITEPACK=env/site-packages
mkdir -p $SITEPACK

mkdir $SITEPACK/ruamel
mkdir $SITEPACK/ruamel/yaml

printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py
printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py
printf '' > $SITEPACK/ruamel/yaml/py.typed

printf 'import ruamel.yaml' > a.py
printf 'import a' > main.py

rm -rf .mypy_cache
PYTHONPATH=$SITEPACK mypy main.py
PYTHONPATH=$SITEPACK mypy main.py

Fixes python#12664

See python#18907 which is another possible fix

I'm not sure why this logic is actually needed given that we should
record the dependency from import analysis in fastparse... maybe just
because it adjusts priority?

I can't write a good test for this because it requires something in site_packages, but here's a minimal repro:
```
set -eux
rm -rf repro
mkdir repro
cd repro

SITEPACK=env/site-packages
mkdir -p $SITEPACK

mkdir $SITEPACK/ruamel
mkdir $SITEPACK/ruamel/yaml

printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py
printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py
printf '' > $SITEPACK/ruamel/yaml/py.typed

printf 'import ruamel.yaml' > a.py
printf 'import a' > main.py

rm -rf .mypy_cache
PYTHONPATH=$SITEPACK mypy main.py
PYTHONPATH=$SITEPACK mypy main.py
```

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

hauntsaninja changed the title Fix incremental issue with namespace packages Fix incremental issue with namespace packages (option 2) Apr 11, 2025
hauntsaninja added a commit that referenced this pull request Apr 11, 2025
Fixes #12664

A root cause is there is this stateful `_update_ns_ancestors` thing in
`modulefinder`, so if things get called in the wrong order, you can get
incorrect results.

See also the logic in `all_imported_modules_in_file` where we've fixed
several bugs like this previously, like #13124 and #10937

As a result of (seemingly accidentally) reusing imports across modules,
we can end up in a situation where the namespace gets added as a
dependency to all other modules and so on the cached run we attempt to
find namespace before package, which does not work

I am not sure this `imports` code path is even needed, so I will open an
alternate PR, see #18908.

Relevant history:
- #6582
- #6179

I can't write a good test for this because it requires something in
site_packages, but here's a minimal repro:
```
set -eux
rm -rf repro
mkdir repro
cd repro

SITEPACK=env/site-packages
mkdir -p $SITEPACK

mkdir $SITEPACK/ruamel
mkdir $SITEPACK/ruamel/yaml

printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py
printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py
printf '' > $SITEPACK/ruamel/yaml/py.typed

printf 'import ruamel.yaml' > a.py
printf 'import a' > main.py

rm -rf .mypy_cache
PYTHONPATH=$SITEPACK mypy main.py
PYTHONPATH=$SITEPACK mypy main.py
```

Copy link
Copy Markdown
Collaborator Author

Going with #18907 on Ivan's recommendation

hauntsaninja deleted the ruamelfix2 branch April 11, 2025 20:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incremental parsing issue with namespace packages

1 participant


Back | FazBrowse Home | New Git URL