Repository navigation
gh-154675: Use lazy imports in pdb, bdb, code - #156567
hugovk merged 2 commits into
Conversation
Sorry, something went wrong.
| import _pyrepl.utils | ||
| except ModuleNotFoundError: | ||
| _pyrepl = None | ||
| lazy import _colorize |
There was a problem hiding this comment.
Hmm, this is now different to the other files, where we have lazy imports directly after the regular imports.
Sorry, something went wrong.
There was a problem hiding this comment.
The isort/ruff rule is:
import a
from b import c
lazy import d
lazy from e import fAnd this is following that, except I left the spaces between groups, but can remove those if you like?
Sorry, something went wrong.
|
I think making some import lazy definitely makes sense. Making import pdb faster benefits more than pdb itself - pytest is a great example. But how did we choose the libraries to lazy import? Is it quite arbitrary? Is there a rule to follow in stdlib - like which are the libraries we should lazy import? Or did we leave the ones that will "definitely" be used? I'm not against the current list, but it might be clearer for future developers if we have a guidance to follow. |
Sorry, something went wrong.
|
I ran: pip install tuna
./python.exe -X importtime -c "import pdb" 2> import.log
tuna import.logThis gives the blue diagram to visualise the import times. Then looked at the slowest imports, and made them lazy one by one. Some would gett reified right away, like bdb (due to class Pdb(bdb.Bdb, cmd.Cmd)) and code (due to class _PdbInteractiveConsole(code.InteractiveConsole)), so I left those as eager imports but also made some of their own imports lazy in the same way to make then lighter. Others like typing also get reified immediately (due to @typing.override), so that was also left as eager. We've tackled making typing faster before. The tests ensure that a lazy import doesn't end up getting immediately reified due to some other change. If that happens down the line, we can either do something to ensure it remains lazy, or make a conscious decision to allow it be eager, and update the test. We also have some deferred imports, like runpy, which are imported inside functions. This was the pre-lazy way of doing it, so we can now hoist them up as explicit lazy imports. |
Sorry, something went wrong.
gaogaotiantian
left a comment
There was a problem hiding this comment.
I don't have a strong opinion on this. If we believe this is beneficial, I'm okay with the approach.
Sorry, something went wrong.
The goal here is to the improve import time of pdb, one of the slowest stdlib modules to import.
Before: 52 ms
main on macOS with optimised build:
After: 9 ms
We can lazily import several dependencies, and also do the same in its dependencies bdb and code to make them lighter:
pdb.set_trace() (or breakpoint() with defaults) is commonly used for this module, and will reify 6 of the 18 lazy imports here, and the other 12 remain deferred.
And pdb is often imported without running set_trace(), so we get the benefit of all 18 lazy imports.
This also speeds up every pytest invocation, not just pytest --pdb: pytest unconditionally imports pdb at startup via its default debugging plugin. The pytest project found deferring import of pdb saved ~7% of pytest startup on Python 3.14 (pytest-dev/pytest#14874), but the change was declined in favour of improving import time in CPython itself.