FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-150942: Speed up re.findall and re.sub/subn result building by eendebakpt · Pull Request #150943 · python/cpython · GitHub

Repository navigation

gh-150942: Speed up re.findall and re.sub/subn result building - #150943

Merged
corona10 merged 2 commits into
python:mainfrom
eendebakpt:re-takeref-opt
Jun 7, 2026
Merged

corona10 merged 2 commits into
python:mainfrom
eendebakpt:re-takeref-opt

Conversation

eendebakpt commented Jun 5, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF, removing a reference-count round-trip per appended item (and a per-append lock on the free-threaded build).

Microbenchmarks

Benchmark main this PR speedup
re_sub 1.93 ms 1.73 ms 1.11×
re_subn 1.92 ms 1.73 ms 1.11×
re_findall 1.89 ms 1.80 ms 1.05×
re_findall_groups 4.01 ms 3.87 ms 1.04×
geomean 1.06×
Benchmark script
"""Microbenchmark re.findall / split / sub / subn (result-list building)."""
import re
import pyperf

WORDS = "the quick brown fox jumps over the lazy dog " * 2000
CSVISH = "field1,field2,field3,field4,field5,field6,field7,field8\n" * 3000

word_re = re.compile(r"\w+")
group_re = re.compile(r"(\w)(\w+)")
split_re = re.compile(r"[,\n]")
sub_re = re.compile(r"\w+")


def b_findall():        return word_re.findall(WORDS)
def b_findall_groups(): return group_re.findall(WORDS)   # list of tuples
def b_sub():            return sub_re.sub("X", WORDS)
def b_subn():           return sub_re.subn("X", WORDS)


if __name__ == "__main__":
    runner = pyperf.Runner()
    runner.bench_func("re_findall", b_findall)
    runner.bench_func("re_findall_groups", b_findall_groups)
    runner.bench_func("re_sub", b_sub)
    runner.bench_func("re_subn", b_subn)

Append result items to the output list with _PyList_AppendTakeRef instead
of PyList_Append followed by Py_DECREF, removing a reference-count
round-trip per appended item (and a per-append lock on the free-threaded
build).  Applied to the result lists built by findall and the sub/subn
helper, where the append is a meaningful share of the work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

sergey-miryanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Looks good to me

corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM. The code looks simpler, and it also brings a performance improvement.

corona10 merged commit 0f7dc2f into python:main Jun 7, 2026
64 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL