gh-118761: substitute `re` import in `base64.b16decode` for a more efficient alternative by picnixz · Pull Request #128736 · python/cpython · GitHub

Benchmarks are on a RELEASE build (no PGO, no LTO).

See #128736 (comment) for the runtime performance improvements as well.

PR

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       179 |        179 | linecache
import time:       205 |        205 |     _struct
import time:       442 |        646 |   struct
import time:       227 |        227 |   binascii
import time:       298 |       1169 | base64

$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       5.7 ms ±   0.5 ms    [User: 4.8 ms, System: 1.1 ms]
  Range (min … max):     5.2 ms …  11.9 ms    455 runs

Main

$ ./python -I -X importtime -c 'import base64'
import time: self [us] | cumulative | imported package
...
import time:       180 |        180 | linecache
import time:       312 |        312 |       types
import time:      1609 |       1921 |     enum
import time:        87 |         87 |       _sre
import time:       250 |        250 |         re._constants
import time:       388 |        637 |       re._parser
import time:        89 |         89 |       re._casefix
import time:       339 |       1150 |     re._compiler
import time:       103 |        103 |         itertools
import time:        83 |         83 |         keyword
import time:        58 |         58 |           _operator
import time:       206 |        264 |         operator
import time:       129 |        129 |         reprlib
import time:        49 |         49 |         _collections
import time:       785 |       1411 |       collections
import time:        43 |         43 |       _functools
import time:       543 |       1997 |     functools
import time:       122 |        122 |     copyreg
import time:       826 |       6015 |   re
import time:       115 |        115 |     _struct
import time:        74 |        189 |   struct
import time:       122 |        122 |   binascii
import time:       279 |       6603 | base64

$ hyperfine --warmup 16 "./python -c 'import base64'"
Benchmark 1: ./python -c 'import base64'
  Time (mean ± σ):       9.5 ms ±   0.4 ms    [User: 8.2 ms, System: 1.3 ms]
  Range (min … max):     9.1 ms …  13.5 ms    295 runs

Issue: Improve import time of various stdlib modules #118761

Importing `base64` is now up to six times faster. The `re` module is now locally imported by `base64.b16decode` and is no more implicitly exposed as `base64.re`.

@hugovk I'm requesting your review since you've commented on the issue. There are also #128732 and #128738 to review FYI.

This entirely removes the needs to a regex.

🚀

…8oADD.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

…8oADD.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

Thanks!

A

@AA-Turner merged commit bbd3300 into python:main Jan 14, 2025

FYI @picnixz was recently promoted as a core dev and so can merge his own changes.

(I don't mind others merging my PRs by the way)

and reduce the import time of :mod:base64 by up to six times.

9.5 ms down to 5.7 ms is impressive but not six times - rather 60%?

hyperfine benchmarks also take into account interpreter's startup and so, while -X importtime only takes into account the time needed to import the module itself (hence the differences).

Sure, my bad. Importtime itself is of course six times faster.

Thank you all for the great work here in cPython!

picnixz added 2 commits January 11, 2025 12:59

Improve import time of the base64 module.

3a92fce

Importing `base64` is now up to six times faster. The `re` module is now locally imported by `base64.b16decode` and is no more implicitly exposed as `base64.re`.

blurb

0b52712

bedevere-app Bot mentioned this pull request Jan 11, 2025

Improve import time of various stdlib modules #118761

Closed

bedevere-app Bot added the awaiting core review label Jan 11, 2025

picnixz added the performance Performance or resource usage label Jan 11, 2025

picnixz requested a review from hugovk January 11, 2025 16:30

AA-Turner reviewed Jan 12, 2025

View reviewed changes

Comment thread Lib/base64.py Outdated

picnixz added 2 commits January 13, 2025 13:23

Improve base64.b16decode() by a factor 10.

d7b6253

This entirely removes the needs to a regex.

update NEWS

9e8fe5d

hugovk approved these changes Jan 13, 2025

View reviewed changes

Comment thread Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f8oADD.rst Outdated

bedevere-app Bot added awaiting merge and removed awaiting core review labels Jan 13, 2025

Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…

5be6178

…8oADD.rst Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

picnixz changed the title gh-118761: improve import time for base64 gh-118761: substitute re import in base64.b16decode for a more efficient alternative Jan 13, 2025

add What's New entry

b36d915

picnixz requested a review from AA-Turner January 14, 2025 12:26

AA-Turner reviewed Jan 14, 2025

View reviewed changes

Comment thread Doc/whatsnew/3.14.rst Outdated

Comment thread Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f8oADD.rst Outdated

picnixz and others added 2 commits January 14, 2025 13:43

Update Doc/whatsnew/3.14.rst

5562c81

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

Update Misc/NEWS.d/next/Library/2025-01-10-13-06-54.gh-issue-118761.f…

82d18d7

…8oADD.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

picnixz mentioned this pull request Jan 14, 2025

gh-118761: improve import time for secrets #128738

Closed

picnixz requested review from AA-Turner and hugovk January 14, 2025 13:19

AA-Turner approved these changes Jan 14, 2025

View reviewed changes

AA-Turner merged commit bbd3300 into python:main Jan 14, 2025

bedevere-app Bot removed the awaiting merge label Jan 14, 2025

picnixz deleted the perf/import/base64-118761 branch January 14, 2025 13:49

picnixz mentioned this pull request Jan 17, 2025

gh-118761: Reduce import time of gettext.py by delaying re import #128898

Merged

gh-118761: substitute `re` import in `base64.b16decode` for a more efficient alternative by picnixz · Pull Request #128736 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-118761: substitute re import in base64.b16decode for a more efficient alternative - #128736

gh-118761: substitute re import in base64.b16decode for a more efficient alternative#128736
AA-Turner merged 8 commits into
python:mainfrom
picnixz:perf/import/base64-118761

picnixz commented Jan 11, 2025 •
edited
Loading

Uh oh!

Uh oh!

picnixz commented Jan 11, 2025

Uh oh!

Uh oh!

hugovk left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AA-Turner commented Jan 14, 2025

Uh oh!

vstinner commented Jan 14, 2025

Uh oh!

picnixz commented Jan 14, 2025

Uh oh!

chris-eibl commented Jan 14, 2025 •
edited
Loading

Uh oh!

Uh oh!

picnixz commented Jan 14, 2025

Uh oh!

chris-eibl commented Jan 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-118761: substitute `re` import in `base64.b16decode` for a more efficient alternative by picnixz · Pull Request #128736 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-118761: substitute re import in base64.b16decode for a more efficient alternative - #128736

gh-118761: substitute re import in base64.b16decode for a more efficient alternative#128736AA-Turner merged 8 commits intopython:mainpython/cpython:mainfrom picnixz:perf/import/base64-118761picnixz/cpython:perf/import/base64-118761Copy head branch name to clipboard

Conversation

picnixz commented Jan 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR

Main

Uh oh!

picnixz commented Jan 11, 2025

Uh oh!

Uh oh!

hugovk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AA-Turner commented Jan 14, 2025

Uh oh!

vstinner commented Jan 14, 2025

Uh oh!

picnixz commented Jan 14, 2025

Uh oh!

chris-eibl commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz commented Jan 14, 2025

Uh oh!

chris-eibl commented Jan 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-118761: substitute re import in base64.b16decode for a more efficient alternative#128736
AA-Turner merged 8 commits into
python:mainfrom
picnixz:perf/import/base64-118761

picnixz commented Jan 11, 2025 •
edited
Loading

chris-eibl commented Jan 14, 2025 •
edited
Loading