gh-123880: Allow recursive import of single-phase-init modules - #123950
Conversation
Sorry, something went wrong.
There was a problem hiding this comment.
Thank you for the fix, confirmed it on the end-to-end mypyc repro as well. I was wondering if we could get away with just removing the assert like this :-) I noticed two trivial typos in comments
Sorry, something went wrong.
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
| loader = importlib.machinery.ExtensionFileLoader(name, filename) | ||
| spec = importlib.util.spec_from_file_location(name, filename, | ||
| loader=loader) | ||
| mod = importlib._bootstrap._load(spec) |
There was a problem hiding this comment.
I don't know if I love using this API for the test since it's somewhat of a hack to start, but I also understand not wanting to paste in 2 more lines of boilerplate to get the module created and set sys.modules.
Sorry, something went wrong.
There was a problem hiding this comment.
It was done elsewhere in the file too; I merged the usage into a common helper. If it breaks, there's now just one place to change.
(We never got around to adding proper public API for multiple modules in one shared library, but then, it's really only useful for testing import mechanism...)
In this case I slightly prefer not to touch sys.modules here -- the “inner” import should do that.
Sorry, something went wrong.
Co-authored-by: Brett Cannon <brett@python.org>
|
!buildbot iOS |
Sorry, something went wrong.
|
!buildbot iOS |
Sorry, something went wrong.
|
Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13. |
Sorry, something went wrong.
…ythonGH-123950) (cherry picked from commit aee219f) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Brett Cannon <brett@python.org>
|
GH-124273 is a backport of this pull request to the 3.13 branch. |
Sorry, something went wrong.
…ythonGH-123950) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Brett Cannon <brett@python.org>
…ythonGH-123950) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Brett Cannon <brett@python.org>
…GH-123950) (#124273) gh-123880: Allow recursive import of single-phase-init modules (GH-123950) (cherry picked from commit aee219f) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Brett Cannon <brett@python.org>
Single-phase modules can imported recursively, if they take care of returning the partially initialized module in such cases.
Concretely, mypyc emits a global static PyObject* module = NULL; set right after PyModule_Create and checked+returned right before it. This is of course utterly incompatible with multiple interpreters and with Py_Finalize/Py_Initialize “power cycles”, but, it works for their use cases.
Unless I'm missing something, the assert that mypyc is running into can be removed, and the cache value reused. Deallocation of the cache values is done in a few places, but none of them can reasonably happen witin a (recursive) import.
(Here I beg people to not try destroying pre-existing interpreters from within a recursive single-phase PyInit_* function -- please! single-phase init is a bowl of backcompat spaghetti; even after Eric's untangling, it can only take so much. See several XXX notes elsewhere in import.c.)
I'm sending this draft early, I still have to check more of the code and write a regression test.