FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-59110: zipimport: support namespace packages when no directory entry exists by serhiy-storchaka · Pull Request #121233 · python/cpython · GitHub

Repository navigation

gh-59110: zipimport: support namespace packages when no directory entry exists - #121233

Merged
serhiy-storchaka merged 5 commits into
python:mainfrom
serhiy-storchaka:zipimport-namespace
Jul 4, 2024
Merged

serhiy-storchaka merged 5 commits into
python:mainfrom
serhiy-storchaka:zipimport-namespace

Conversation

serhiy-storchaka commented Jul 1, 2024 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

serhiy-storchaka changed the title gh-121111: zipimport: fix support of namespaces containing only subpackages gh-121111:zipimport: fix support of namespaces containing only subpackages Jul 1, 2024
serhiy-storchaka changed the title gh-121111:zipimport: fix support of namespaces containing only subpackages gh-121111: zipimport: fix support of namespaces containing only subpackages Jul 1, 2024

ZeroIntensity commented Jul 1, 2024 •
edited
Loading

Copy link
Copy Markdown
Member

Related to #121161, which fixes the other half of the issue (the test that I wrote there does not pass on this branch). @AraHaan can you confirm if this patch fixes #121111?

Comment thread Lib/test/test_zipimport.py Outdated
self.assertEqual(spec.submodule_search_locations,
[(TEMP_ZIP + "/a/b").replace("/", zipimport.path_sep)])
self.assertRaises(zipimport.ZipImportError, importer.get_code, "a.b")
self.assertEqual(importer.get_data("a/b/"), b"")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't know if this is a necessary test case for get_data, maybe directory path will be never passed to it.

SeaHOH commented Jul 1, 2024

Copy link
Copy Markdown
UNEXPECTED SUCCESS: test_missing_directory (test.test_importlib.test_namespace_pkgs.ZipWithMissingDirectory.test_missing_directory)

Interesting, this was not a bug before, someone who wrote the test knows how it is working.

Comment thread Lib/test/test_zipimport.py Outdated
Comment on lines +463 to +472
try:
import a.b.c
self.assertIn('namespace', str(a.b).lower())
self.assertEqual(a.b.c.foo(), "foo")
finally:
del sys.path[0]
sys.modules.pop('a.b.c', None)
sys.modules.pop('a.b', None)
sys.modules.pop('a', None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I see the ImportHooksBaseTestCase has setUp and tearDown to restore the environ, so the finally section could be removed.

AraHaan commented Jul 4, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

I did a little test and it seems that this change also fixed my use case as well.

serhiy-storchaka changed the title gh-121111: zipimport: fix support of namespaces containing only subpackages gh-59110: zipimport: support namespace packages when no directory entry exists Jul 4, 2024

Copy link
Copy Markdown
Member Author

Changed the reference to the older issue #59110.

serhiy-storchaka enabled auto-merge (squash) July 4, 2024 14:48
serhiy-storchaka merged commit 17d5b9d into python:main Jul 4, 2024
Comment thread Lib/zipimport.py
serhiy-storchaka deleted the zipimport-namespace branch July 4, 2024 17:28
Comment thread Lib/zipimport.py
Comment on lines +561 to +570
for name in list(files):
while True:
i = name.rstrip(path_sep).rfind(path_sep)
if i < 0:
break
name = name[:i + 1]
if name in files:
break
files[name] = None
count += 1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I probably would have written this as something like:

    files.update(dict.fromkeys(zipfile._path.CompleteDirs._implicit_dirs(files)))

(or some variation that ports the CompleteDirs._implicit_dirs without itertools). The advantages of using that implementation are:

  • it's a proven implementation and has already encountered real-world edge cases
  • it uses lazy evaluation to ensure compact memory usage
  • it applies functional principles, avoiding mutating structures in place and simplifying the logic

If you want the count of files added:

    updates = dict.fromkeys(zipfile._path.CompleteDirs._implicit_dirs(files))
    files.update(updates)
    count = len(updates)

Of course, CompleteDirs operates on posixpath.sep and files here uses pathsep for separators, so maybe it's not worth porting CompleteDirs. Still, I think it would be nice if this new behavior were factored out so as to limit making this already enormous function much larger.

I set out to apply this approach in #121378.

noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024

jaraco commented Sep 9, 2024

Copy link
Copy Markdown
Member

This change feels like a bugfix. The bug is affecting users of Setuptools in pypa/setuptools#4641. Could the change be backported to bugfix Pythons?

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants


Back | FazBrowse Home | New Git URL