FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Free-threading: QSBR thread states false-share on 128-byte cache lines (Apple Silicon), CPU-bound threads reach ~6× on 14 cores where processes reach ~10× · Issue #159860 · python/cpython · GitHub

Repository navigation

Free-threading: QSBR thread states false-share on 128-byte cache lines (Apple Silicon), CPU-bound threads reach ~6× on 14 cores where processes reach ~10× #159860

Open

Description

Bug report

Hey I'm the Cinema 4D dev responsible for the Python integration. While testing the Free-Threaded version of CPython 3.15 I noticed poor scaling in my mac. So I let IA run some iterations and it eventually found some false cache sharing. Here is the report written by IA (I prefer to be honest since I do not know exactly the policy of this repo regarding IA).

Bug report

Bug description

On Apple Silicon, free-threaded threads doing CPU-bound work that share no objects reach about half the speedup of processes. Most of Tools/ftscalingbench scales only 2.3–4× on 14 cores. 3.13, 3.14, 3.15 and main all behave the same.

M4 Pro (10P + 4E cores), ./configure --disable-gil:

3.13.16 3.14.8 3.15.0 main main, padded to 128
ftscalingbench.py -t 10 --scale 10000 generator --parallel-only (from gh-144438) 706–711 ms 631–636 ms 668–687 ms 476–584 ms 258–265 ms
pure-Python loop, 14 threads, speedup over 1 thread 6.1–6.4× 5.8–6.7× 5.9–6.2× 5.3–6.4× 10.1–10.3×
same loop, 14 processes 10.4×

Cause: _Py_qsbr_quiescent_state() stores into the thread's QSBR state on every JUMP_BACKWARD and CALL. The states of all threads are in one array of struct _qsbr_pad, padded to 64 bytes, the x86 line size. Apple Silicon lines are 128 bytes (sysctl hw.cachelinesize → 128), so two threads share each line and fight over it on every loop iteration. gh-144438 also used 64 bytes, for its array alignment and its _PyThreadStateImpl padding, so it doesn't help here.

Why 128 bytes

  • Apple's hardware: macOS reports hw.cachelinesize: 128. LLVM changed clang's destructive interference size for Apple AArch64 to 128 in llvm/llvm-project#208716, saying "Apple arm64 cores use a 128-byte cache line" (code).
  • Measured: in a C loop that does exactly what _Py_qsbr_quiescent_state() does, 14 threads take 0.64–0.80 ns per iteration with 64-byte slots, 0.33–0.38 ns with 128-byte slots, and 0.34–0.35 ns with 256-byte slots (qsbr_stride.c below). So 128 is necessary, and larger gains nothing.
  • Other runtimes chose 128 on Apple Silicon / AArch64 for the same reason:
  • Cost: about 64 bytes per QSBR slot, on AArch64 only (details in the PR).

Reproducer

git clone https://github.com/python/cpython.git && cd cpython
git checkout d42178a811d0          # main on 2026-10-19; v3.13.16 / v3.14.8 / v3.15.0 behave the same
./configure --disable-gil && make -j8
./python.exe Tools/ftscalingbench/ftscalingbench.py -t 10 --scale 10000 generator --parallel-only
./python.exe Tools/ftscalingbench/ftscalingbench.py      # all benchmarks, os.cpu_count() threads
./python.exe qsbr_false_sharing.py                       # threads vs processes (script below)
cc -O2 -o qsbr_stride qsbr_stride.c && ./qsbr_stride     # the same store pattern in plain C (below)
qsbr_false_sharing.py
"""CPU-bound pure-Python work: free-threaded threads vs. processes.

Every task runs the same loop and touches no shared Python object, so threads
should scale like processes. --interleave starts an idle (detached) thread
after every worker, so no two busy threads get neighbouring QSBR slots.
"""
import argparse
import multiprocessing
import os
import queue
import sys
import threading
import time
from concurrent.futures import ProcessPoolExecutor

LOOP = 200_000
TASKS = 256


def work(_=None):
    acc = 0
    for i in range(LOOP):
        acc += i & 7
    return acc


def run_threads(workers, interleave):
    tasks = queue.SimpleQueue()
    for _ in range(TASKS):
        tasks.put(None)
    go = threading.Event()
    stop = threading.Event()

    def worker():
        go.wait()
        while True:
            try:
                tasks.get_nowait()
            except queue.Empty:
                return
            work()

    busy, idle = [], []
    for _ in range(workers):
        t = threading.Thread(target=worker)
        t.start()
        busy.append(t)
        if interleave:
            t = threading.Thread(target=stop.wait)
            t.start()
            idle.append(t)
    time.sleep(0.05)  # let every thread reach its wait
    t0 = time.perf_counter()
    go.set()
    for t in busy:
        t.join()
    elapsed = time.perf_counter() - t0
    stop.set()
    for t in idle:
        t.join()
    return elapsed


def run_processes(workers):
    ctx = multiprocessing.get_context("spawn")
    with ProcessPoolExecutor(workers, mp_context=ctx) as pool:
        list(pool.map(work, range(2 * workers)))  # start + warm up
        t0 = time.perf_counter()
        list(pool.map(work, range(TASKS), chunksize=1))
        return time.perf_counter() - t0


def best(fn, *args, repeat=3):
    return min(fn(*args) for _ in range(repeat))


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--workers", default=None,
                        help="comma-separated worker counts (default: 1,4,8,...,cpu_count)")
    parser.add_argument("--interleave", action="store_true",
                        help="start an idle thread between the worker threads")
    parser.add_argument("--no-processes", action="store_true")
    args = parser.parse_args()

    ncpu = os.cpu_count()
    counts = ([int(n) for n in args.workers.split(",")] if args.workers
              else sorted({1, 4, *range(8, ncpu + 1, 2), ncpu}))
    print(sys.version)
    print("GIL enabled:", sys._is_gil_enabled() if hasattr(sys, "_is_gil_enabled") else True,
          "| cpu_count:", ncpu, "| tasks:", TASKS, "x", LOOP, "iterations")

    rows = [("threads" + (" (interleaved)" if args.interleave else ""),
             lambda n: best(run_threads, n, args.interleave))]
    if not args.no_processes:
        rows.append(("processes", lambda n: best(run_processes, n)))
    for name, fn in rows:
        t1 = fn(1)
        speedups = " ".join(f"{n}:{t1 / fn(n):4.1f}x" for n in counts)
        print(f"{name:<24} 1 worker {t1 * 1000:6.0f} ms   speedup  {speedups}", flush=True)


if __name__ == "__main__":
    main()
qsbr_stride.c
// What _Py_qsbr_quiescent_state() does, without CPython: each thread reads a
// shared sequence (acquire) and stores it into its own slot (release).
// Slots are 64, 128 or 256 bytes apart, like struct _qsbr_pad.
//   cc -O2 -o qsbr_stride qsbr_stride.c && ./qsbr_stride
#include <pthread.h>
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>

#define ITERS 50000000L
#define MAX_THREADS 64

static _Atomic uint64_t wr_seq = 1;
static char *slots;
static size_t stride;
static int skip_unchanged;
static _Atomic int go;

static void *worker(void *arg)
{
    _Atomic uint64_t *seq = (_Atomic uint64_t *)(slots + (size_t)arg * stride);
    while (!atomic_load_explicit(&go, memory_order_acquire)) {
    }
    for (long i = 0; i < ITERS; i++) {
        uint64_t s = atomic_load_explicit(&wr_seq, memory_order_acquire);
        if (skip_unchanged && atomic_load_explicit(seq, memory_order_relaxed) == s) {
            continue;
        }
        atomic_store_explicit(seq, s, memory_order_release);
    }
    return NULL;
}

static double run(int nthreads)
{
    pthread_t t[MAX_THREADS];
    struct timespec a, b;
    memset(slots, 0, MAX_THREADS * 256);
    atomic_store(&go, 0);
    for (long i = 0; i < nthreads; i++) {
        pthread_create(&t[i], NULL, worker, (void *)i);
    }
    usleep(20000);
    clock_gettime(CLOCK_MONOTONIC, &a);
    atomic_store_explicit(&go, 1, memory_order_release);
    for (int i = 0; i < nthreads; i++) {
        pthread_join(t[i], NULL);
    }
    clock_gettime(CLOCK_MONOTONIC, &b);
    return ((b.tv_sec - a.tv_sec) * 1e9 + (b.tv_nsec - a.tv_nsec)) / ITERS;
}

int main(void)
{
    long ncpu = sysconf(_SC_NPROCESSORS_ONLN);
    if (ncpu > MAX_THREADS) ncpu = MAX_THREADS;
    if (posix_memalign((void **)&slots, 256, MAX_THREADS * 256)) return 1;
    int counts[] = {1, 2, 4, 8, (int)ncpu};
    struct { size_t stride; int skip; const char *name; } cfg[] = {
        {64, 0, "64-byte slots"},
        {128, 0, "128-byte slots"},
        {256, 0, "256-byte slots"},
        {64, 1, "64-byte slots, skip unchanged store"},
    };
    printf("ns per iteration (wall), %ld iterations per thread\n", ITERS);
    for (size_t c = 0; c < sizeof(cfg) / sizeof(cfg[0]); c++) {
        stride = cfg[c].stride;
        skip_unchanged = cfg[c].skip;
        printf("%-38s", cfg[c].name);
        for (size_t k = 0; k < sizeof(counts) / sizeof(counts[0]); k++) {
            double best = 1e30;
            for (int r = 0; r < 3; r++) {
                double v = run(counts[k]);
                if (v < best) best = v;
            }
            printf("  %2d thr: %5.2f", counts[k], best);
        }
        printf("\n");
    }
    return 0;
}

CPython versions tested on

3.13, 3.14, 3.15, CPython main branch

Operating systems tested on

macOS

Linked PRs

Activity

  1. sergey-miryanov commented on Oct 20, 2026

    Contributor
  2. added 3 commits that reference this issue on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions


      Back | FazBrowse Home | New Git URL