FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-159860: Adjust QSBR cache line size from 64 to 128 on AArch64 by gr4ph0s · Pull Request #159862 · python/cpython · GitHub

Repository navigation

gh-159860: Adjust QSBR cache line size from 64 to 128 on AArch64 - #159862

Merged
corona10 merged 5 commits into
python:mainfrom
gr4ph0s:gh-159860-qsbr-128
Oct 20, 2026
Merged

corona10 merged 5 commits into
python:mainfrom
gr4ph0s:gh-159860-qsbr-128

Conversation

gr4ph0s commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Adds _Py_QSBR_CACHE_LINE_SIZE (128 on __aarch64__ / _M_ARM64, 64 elsewhere) and uses it in the three places that hard-code 64:

before after (AArch64)
struct _qsbr_pad char __padding[64 - sizeof(...)] 128-byte slots
QSBR array alignment in grow_thread_array() (gh-144438) 64 128
_PyThreadStateImpl tail padding (gh-144438) char __padding[64] char __padding[128]

The change is internal only (Py_BUILD_CORE): no ABI change, and other architectures are unchanged.

What goes wrong today

  1. check_periodics() / check_periodics_at_end() run on every JUMP_BACKWARD, after every CALL, and in RESUME. Each time they call _Py_qsbr_quiescent_state(), which does an unconditional _Py_atomic_store_uint64_release(&qsbr->seq, ...).
  2. sizeof(struct _qsbr_thread_state) is already 64 (checked with lldb), so _qsbr_pad adds 0 bytes of padding. On 128-byte lines, slots 2k and 2k+1 share a line.
  3. Threads created one after another get neighbouring slots. initialize_new_array() pushes the free slots onto a LIFO freelist in index order, and _Py_qsbr_reserve() pops from it. Pool workers therefore end up pairwise on shared lines.

To check that the slot layout is the cause: an idle (detached) thread started after each worker keeps busy threads off neighbouring slots. On unpatched main that alone raises 14-thread scaling from ~6× to 7.9–8.6×.

Why gh-144438 didn't cover it

gh-144438 was found on x86, with 64-byte lines. The problem there was the first array entry sharing a line with some thread's tlbc_index (diagnosis). Its fix uses 64 for both the array alignment and the new tail padding. On x86 a 64-byte _qsbr_pad already gives each thread its own line, so the padding between entries never needed changing. On 128-byte lines it does, and the gh-144438 padding is only half a line. The layout is unchanged since 3.13 (3.13.16), so on a Mac the penalty is the same in every version. This fits why the gh-144438 regression was "not seen on Mac" (comment).

Results

M4 Pro (10P + 4E), macOS 26.7.1, Apple clang 17, ./configure --disable-gil, base main d42178a.

main this PR
ftscalingbench.py -t 10 --scale 10000 generator --parallel-only 476–584 ms 258–265 ms
ftscalingbench.py (14 threads), e.g. staticmethod_call / classmethod_call / pymethod 2.4× / 2.3× / 3.4× 7.9× / 8.0× / 7.5–8.1×
pure-Python loop, 14 threads (14 processes: 10.4×) 5.3–6.4× 10.1–10.3×
same loop, 1 thread 840–901 ms 843–884 ms
All ftscalingbench benchmarks (14 threads, two runs)
benchmark main this PR
object_cfunction 3.4 / 3.5 7.9 / 8.1
cmodule_function 2.5 / 3.7 7.5 / 8.3
object_lookup_special 7.6 / 7.0 7.6 / 8.5
context_manager 3.9 / 4.2 7.9 / 8.0
mult_constant 2.6 / 3.4 7.9 / 8.5
generator 2.4 / 6.7 6.7 / 8.1
pymethod 3.4 / 3.3 7.5 / 8.1
pyfunction 4.1 / 3.6 7.5 / 3.8
module_function 3.6 / 3.8 8.0 / 8.3
load_string_const 3.1 / 4.2 8.0 / 8.1
load_tuple_const 3.6 / 4.2 8.5 / 8.6
create_pyobject 4.3 / 5.7 8.8 / 8.9
create_closure 4.3 / 6.1 8.9 / 8.6
create_dict 4.0 / 4.1 8.0 / 7.4
create_frozendict 5.0 / 3.6 8.3 / 8.2
thread_local_read 5.6 / 5.9 6.3 / 7.5
method_caller 5.9 / 4.8 6.8 / 8.1
instantiate_dataclass 7.9 / 7.4 9.8 / 8.6
instantiate_namedtuple 7.3 / 6.2 8.2 / 9.7
instantiate_typing_namedtuple 7.1 / 6.9 9.9 / 10.3
super_call 3.7 / 3.4 7.8 / 7.7
classmethod_call 2.3 / 2.4 8.0 / 7.8
staticmethod_call 2.4 / 2.3 7.9 / 7.9
descriptor 3.3 / 3.3 8.2 / 8.8
deepcopy 3.3 / 3.4 3.3 / 3.3
setattr_non_interned 5.9 / 6.1 9.8 / 9.6
enum_attr 4.2 / 4.4 6.7 / 8.0
type_lookup 4.1 / 4.5 4.1 / 3.7

deepcopy and type_lookup are limited by something else.

Pure-Python loop per release and worker count; C stride test

Speedup over 1 worker (qsbr_false_sharing.py from the issue):

build 4 8 10 12 14
v3.13.16 3.7–3.9× 5.4–5.8× 7.9–8.2× 6.3–6.6× 6.1–6.4×
v3.14.8 3.6–3.8× 5.6× 5.1–5.9× 6.0–6.7× 5.8–6.7×
v3.15.0 3.6–3.8× 5.4–6.1× 5.2–8.2× 6.0–7.2× 5.9–6.2×
main 3.3–4.0× 5.1–6.1× 5.9–7.9× 5.3–6.0× 5.3–6.4×
main, idle thread between workers 3.7–3.8× 6.5× 7.2–9.1× 8.0–8.8× 7.9–8.6×
main, processes 4.0× 7.6× 9.3× 10.0× 10.4×
this PR 3.7× 7.2–7.3× 8.8–8.9× 9.5–9.6× 10.1–10.3×

There is no difference with 2 threads (1.8–2.0× in every build); the penalty starts at 4.

qsbr_stride.c (the same store pattern in C), ns per iteration. The 14-thread column includes the E-cores.

slots 1 2 4 8 14
64 B 0.22–0.24 0.23–0.24 0.26–0.38 0.42–0.54 0.64–0.80
128 B 0.22–0.24 0.23–0.24 0.24–0.26 0.26 0.33–0.38
256 B 0.22 0.23 0.26 0.26 0.34–0.35
64 B, store skipped if unchanged 0.33–0.36 0.34–0.36 0.37–0.38 0.39 0.51–0.53

Memory cost (AArch64 only)

Measured on a main build with lldb (sizeof, and mimalloc's mi_good_size(), which PyMem_RawCalloc uses in the free-threaded build):

before after
QSBR slot 64 B 128 B
QSBR array, initial 8 slots (per interpreter) 640 B 1280 B
QSBR array, 1024 slots 80 KiB 144 KiB
_PyThreadStateImpl 15,800 B → 16 KiB size class 15,864 B → same 16 KiB size class (+0)

That is about 64 B per QSBR slot. For comparison, CPython reserves a 16 MiB stack per thread on macOS.

Design notes

  • Why 128, and not the compiler's __GCC_DESTRUCTIVE_SIZE:
    • That macro varies by compiler: 64 with Apple clang 17, 128 for Apple targets in upstream clang (llvm/llvm-project#208716), and 256 for generic AArch64 in clang (AArch64.h) and GCC (aarch64.cc).
    • It is documented as ABI-unstable, and MSVC doesn't define it.
    • 256 measures the same as 128 (table above).
    • Go, Rust std, OpenJDK, folly and Blender use 128 (links in the issue).
  • Why all of AArch64 and not only __APPLE__: on AArch64 cores with 64-byte lines, this only costs the memory above, and it also covers Windows on ARM. Easy to narrow if preferred.
  • x86_64 is not changed: Scaling regression in Tools/ftscalingbench/ftscalingbench.py generator #144438 showed 64 is enough there. (Rust std and folly use 128 on x86_64 because of Intel's adjacent-line prefetcher; I couldn't measure x86.)
  • Alternative tried: skip the store when seq is unchanged. It is safe, because only the owner writes seq while attached. It also removes the contention, but it adds a load and a branch to the hottest path: about +50% per call in the C test, and up to ~3% on the 1-thread loop. With the padding it brings nothing extra (10.2–10.3× vs. 10.1–10.3×), so I left it out.

Tests

./python.exe -m test -j12 test_free_threading test_threading test_thread test_queue test_weakref test_gc test_sys test_capi test_concurrent_futures test_threading_local test_itertools test_generators: SUCCESS (79 files, 3256 tests).

🤖 Generated with Claude Code

gr4ph0s and others added 4 commits October 20, 2026 08:21
Apple Silicon has 128-byte cache lines. struct _qsbr_pad was padded to 64
bytes, so the QSBR states of two neighbouring threads shared a cache line and
the unconditional store in _Py_qsbr_quiescent_state() (eval loop back-edges
and calls) made threads that share no Python objects contend on it.

Use _Py_QSBR_CACHE_LINE_SIZE (128 on AArch64, 64 elsewhere) for the padding,
the array alignment and the tail padding of _PyThreadStateImpl added in
pythongh-144438.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

python-cla-bot Bot commented Oct 20, 2026 •
edited
Loading

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

corona10 changed the title gh-159860: Pad QSBR thread states to 128 bytes on AArch64 gh-159860: Adjust QSBR cache line size from 64 to 128 on AArch64 Oct 20, 2026
Comment thread Include/internal/pycore_qsbr.h Outdated
Comment on lines +72 to +77
// Size used to pad and align the per-thread QSBR states (and the end of
// _PyThreadStateImpl) to avoid false sharing. Apple Silicon has 128-byte cache
// lines; with 64-byte padding the `seq` fields of two threads share a line and
// the stores in _Py_qsbr_quiescent_state() contend on every eval loop
// back-edge. Use 128 bytes for all AArch64 targets: on cores with 64-byte
// lines this only costs 64 bytes per thread.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality
Suggested change
// Size used to pad and align the per-thread QSBR states (and the end of
// _PyThreadStateImpl) to avoid false sharing. Apple Silicon has 128-byte cache
// lines; with 64-byte padding the `seq` fields of two threads share a line and
// the stores in _Py_qsbr_quiescent_state() contend on every eval loop
// back-edge. Use 128 bytes for all AArch64 targets: on cores with 64-byte
// lines this only costs 64 bytes per thread.
// Exact cache line size for Apple Silicon should be 128.
// $ sysctl -a hw | grep cachelinesize
// hw.cachelinesize: 128

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Also looks good to me. Just to get super bikesheddy I appreciate some of what the initial comment said ("Size used to pad and align the per-thread QSBR states (and the end of _PyThreadStateImpl) to avoid false sharing. Apple Silicon has 128-byte cache lines" but also really like the "sysctl -a hw | grep cachelinesize" and associated output.

github-project-automation Bot moved this to Todo in Sprint Oct 20, 2026
github-project-automation Bot moved this from Todo to In Progress in Sprint Oct 20, 2026

DinoV left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

awesome!

Comment thread Include/internal/pycore_qsbr.h Outdated
corona10 enabled auto-merge (squash) October 20, 2026 18:17
corona10 merged commit 39bc958 into python:main Oct 20, 2026
56 checks passed
github-project-automation Bot moved this from In Progress to Done in Sprint Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL