gh-144384: Lazily import `_colorize` by hugovk · Pull Request #149318 · python/cpython · GitHub

Lazily import _colorize to avoid slow construction/import when not needed.

The easily testable:

Before	After
difflib: 10ms	2ms (80% faster)

doctest: 48ms	40ms (17% faster)

json.tool: 13ms	6ms (54% faster)

pdb: 43ms	37ms (14% faster)

traceback: 15ms	8ms (47% faster)

unittest: 20ms	14ms (30% faster)

Some indirect improvements:

Before	After
asyncio: 47ms	27ms (43% faster)

logging: 17ms	10ms (41% faster)

traceback needs some extra handling: add a shutdown theme, so if it attempts to reify _colorize during shutdown when the import machinery is no longer around, it has a no-op fallback.

Also add ensure_lazy_imports tests.

Issue: _colorize module is slow to import, affecting traceback and logging modules #144384

Can we have a convention for lazy imports

Can we have a convention for lazy imports

Yes, given we don't have a convention, and the SC left it the details to linters and auto-formatters, let's generally follow the Ruff/isort convention:

import json
import os
import subprocess
from collections import defaultdict
from pathlib import Path
from typing import Final
lazy import ast
lazy import shutil
lazy from dataclasses import dataclass

Seeing #149321, I think this should wait till that is somewhat figured out.

Seeing #149321, I think this should wait till that is somewhat figured out.

#149338 has been merged, updating.

And both the commands from #149321 are working.

The CIFuzz failures are unrelated network errors:

413.7 E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/t/tcltk-defaults/tcl-dev_8.6.9+1_amd64.deb  Connection failed [IP: 185.125.190.81 80]

There's another outage: https://status.canonical.com/#/incident/KNms6QK9ewuzz-7xUsPsNylV20jEt5kyKsd8A-3ptQG1OfBTMZO_fBkVMmGqQC3NLBLamiI6KOEL7KpsobFJ4Q==

CIFuzz now also passing.

Looks good to me, but I'm not very familiar with lazy imports.

I'm not sure this makes sense for the pyrepl modules. Aren't those essentially always going to want _colorize?

I'm not sure the sqlite.__main__ lazy import is worthwhile? While it's not imported at top level, main will immediately import it with ArgumentParser() or get_theme a little later.

I'm not sure this makes sense for the pyrepl modules. Aren't those essentially always going to want _colorize?

NO_COLOR=1 ./python.exe is a thing, and gives you a colour-free REPL.

But it still gets imported:

❯ NO_COLOR=1 ./python.exe
Python 3.15.0a8+ (heads/3.15-lazy-_colorize:2d346d5a1ea, May  6 2026, 16:40:10) [Clang 17.0.0 (clang-1700.6.4.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> "_colorize" in sys.modules
True

So I'll remove that change.

And reverted in sqlite3.__main__ as well.

Documentation build overview

📚 cpython-previews | 🛠️ Build #32566767 | 📁 Comparing ed1f8e6 against main (ffa4d47)

🔍 Preview build

11 files changed · ± 11 modified

± Modified

Also reverted pdb, it imports it via pyrepl:

Yes, pdb also seemed questionable to me since if you use pdb you'll almost certainly want the prompt, which will give you colors.

json.tool is also unnecessary, it uses colorize in main() which is the only interface to that module.

I haven't looked much into profiling, are there plausible code paths where that is being used that don't go through colorize? Same for unittest and doctest.

difflib and traceback seem OK, they're broad libraries and there are plenty of functions in there that don't need color.

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

A couple still seem questionable but the cost of an extra lazy import isn't that high, so I'm OK with this going in.

Thanks all!

hugovk added 4 commits May 3, 2026 16:13

Lazy import _colorize in traceback with shutdown fallback

e5559c2

Lazy import _colorize

5d16d31

Add tests to ensure lazy imports are lazy and fix sqlite3's main

c31b1a5

Add news

6a88c2d

hugovk requested a review from berkerpeksag as a code owner May 3, 2026 14:32

hugovk added the performance Performance or resource usage label May 3, 2026

hugovk requested review from erlend-aasland, lysnikolaou and pablogsal as code owners May 3, 2026 14:32

hugovk added the stdlib Standard Library Python modules in the Lib/ directory label May 3, 2026

hugovk requested review from ambv and gaogaotiantian as code owners May 3, 2026 14:32

bedevere-app Bot mentioned this pull request May 3, 2026

_colorize module is slow to import, affecting traceback and logging modules #144384

Open

bedevere-app Bot added the awaiting core review label May 3, 2026

picnixz reviewed May 3, 2026

View reviewed changes

Comment thread Lib/_pyrepl/fancycompleter.py Outdated

Comment thread Lib/traceback.py

Comment thread Lib/traceback.py

Use mypy on 3.15 to support lazy keyword

1d36600

ZeroIntensity mentioned this pull request May 3, 2026

What is supposed to work under -X lazy_imports=none? #149321

Closed

List lazy later

9fc265e

hugovk requested review from AA-Turner, ezio-melotti and webknjaz as code owners May 3, 2026 15:32

hugovk commented May 3, 2026

View reviewed changes

Comment thread Lib/traceback.py

Comment thread Lib/traceback.py

picnixz reviewed May 3, 2026

View reviewed changes

Comment thread Lib/traceback.py

hugovk added 2 commits May 3, 2026 18:48

Move shutdown theme below __all__

5990755

Merge branch 'main' into 3.15-lazy-_colorize

dfe32c5

Merge branch 'main' into 3.15-lazy-_colorize

3a6306c

Merge remote-tracking branch 'upstream/main' into 3.15-lazy-_colorize

2d346d5

sergey-miryanov approved these changes May 6, 2026

View reviewed changes

hugovk added 3 commits May 6, 2026 16:43

Revert 'lazy import _colorize' in pyrepl

82e17cf

Revert 'lazy import _colorize' in pyrepl

be0b1e3

Revert 'lazy import _colorize' in sqlite3.__main__

0bb0956

hugovk added 2 commits May 6, 2026 17:14

Revert 'lazy import _colorize' in sqlite3.__main__

7bcfc9c

Revert 'lazy import _colorize' in pdb, it imports via _pyrepl

85588e1

JelleZijlstra requested changes May 6, 2026

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting core review labels May 6, 2026

Revert 'lazy import _colorize' in json.tool

ed1f8e6

JelleZijlstra approved these changes May 6, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting changes labels May 6, 2026

hugovk enabled auto-merge (squash) May 6, 2026 16:00

hugovk merged commit 7cea70e into python:main May 6, 2026
67 of 68 checks passed

hugovk deleted the 3.15-lazy-_colorize branch May 6, 2026 16:07

bedevere-app Bot removed the awaiting merge label May 6, 2026

ByteFlowing1337 mentioned this pull request May 20, 2026

gh-150072: Optimize logging imports by lazily importing traceback #150073

Open

hugovk mentioned this pull request Jun 17, 2026

Use lazy imports on Python 3.15 to improve startup speed jazzband/tablib#645

Merged

edvilme mentioned this pull request Jul 23, 2026

gh-154335: Allow disabling terminal colors in Tachyon's pstats_collector module #154344

Open

gh-144384: Lazily import `_colorize` by hugovk · Pull Request #149318 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-144384: Lazily import _colorize - #149318

gh-144384: Lazily import _colorize#149318hugovk merged 16 commits intopython:mainpython/cpython:mainfrom hugovk:3.15-lazy-_colorizehugovk/cpython:3.15-lazy-_colorizeCopy head branch name to clipboard

Conversation

hugovk commented May 3, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hugovk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

StanFromIreland commented May 3, 2026

Uh oh!

hugovk commented May 4, 2026

Uh oh!

hugovk commented May 4, 2026

Uh oh!

hugovk commented May 6, 2026

Uh oh!

sergey-miryanov left a comment

Choose a reason for hiding this comment

Uh oh!

JelleZijlstra commented May 6, 2026

Uh oh!

DavidCEllis commented May 6, 2026

Uh oh!

hugovk commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugovk commented May 6, 2026

Uh oh!

read-the-docs-community Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

hugovk commented May 6, 2026

Uh oh!

JelleZijlstra commented May 6, 2026

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented May 6, 2026

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hugovk commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-144384: Lazily import _colorize#149318
hugovk merged 16 commits into
python:mainfrom
hugovk:3.15-lazy-_colorize

hugovk commented May 3, 2026 •
edited by bedevere-app Bot
Loading

hugovk commented May 6, 2026 •
edited
Loading

read-the-docs-community Bot commented May 6, 2026 •
edited
Loading