Repository navigation
gh-148286: Fix undefined behaviour in per-thread refcount merging - #154915
Open
matthiasgoergens wants to merge 1 commit into
Open
matthiasgoergens wants to merge 1 commit into
matthiasgoergens wants to merge 1 commit into
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
matthiasgoergens
force-pushed
the
freethreaded-refcount-ub
branch
from
91b6f04 to
a4cda41
Compare
July 30, 2026 13:21
…ng (free-threaded)
_PyObject_MergePerThreadRefcounts() merges each thread's local reference
count deltas into the shared count:
_Py_atomic_add_ssize(&obj->ob_ref_shared,
refcnt << _Py_REF_SHARED_SHIFT);
`refcnt` is a delta, so it is routinely negative -- the observed value is
almost always -1 -- and shifting a negative value left is undefined
behaviour. Do the shift in the unsigned domain and convert back.
This is not reachable from the UBSan CI job today: the sanitizer matrix in
.github/workflows/build.yml pairs UBSan only with free-threading: false, so
this file is never built under UBSan. Building --disable-gil with
--with-undefined-behavior-sanitizer shows how load-bearing it is; the very
first line of output from `./python -c pass` is the UBSan report, raised
from interpreter startup via _PyImport_InitExternal, and it recurs from
gc_collect_main on every collection.
Measured over the full test suite on that configuration, with all entries
in Tools/ubsan/suppressions.txt disabled:
before: 1762 UB reports, 529 test failures across 61 test files
after: 0 UB reports, 0 failures, suite green (run=51,517)
Most of those failures are tests that assert a subprocess produced no
stderr, which the UBSan diagnostic breaks.
Adding free-threading: true to the UBSan matrix would keep this fixed; that
is left as a separate decision since it costs a CI job.
matthiasgoergens
force-pushed
the
freethreaded-refcount-ub
branch
from
a4cda41 to
d94417e
Compare
August 5, 2026 08:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_PyObject_MergePerThreadRefcounts() merges each thread's local reference count deltas into the shared count:
refcnt is a delta, so it is routinely negative — in practice almost always -1 — and shifting a negative value left is undefined behaviour. This does the shift in the unsigned domain and converts back.
This is not reachable from the UBSan CI job today. The sanitizer matrix in .github/workflows/build.yml pairs UBSan only with free-threading: false, so Python/uniqueid.c is never built under UBSan at all. Building --disable-gil together with --with-undefined-behavior-sanitizer shows how load-bearing it is: the first line of output from ./python -c pass is the UBSan report, raised during interpreter startup via _PyImport_InitExternal, and it recurs from gc_collect_main on every collection.
Measured over the full test suite on that configuration, with every entry in Tools/ubsan/suppressions.txt disabled:
Most of those failures are tests asserting that a subprocess produced no stderr, which the UBSan diagnostic breaks.
I have not touched the CI matrix here, since adding a job is a maintainer's call and this fix stands on its own. But the reason this went unnoticed is structural rather than accidental, and I would like to follow up separately with a proposal to add free-threading: true to the UBSan matrix — with this fix in, that configuration is green, so it would be a regression guard rather than a source of new work. The same appears to be true of --enable-experimental-jit, which CI builds but never sanitizes: I ran the full suite against a machine-code JIT build under UBSan and it is clean today, while gh-139269 (an unaligned uint64_t store in Python/jit.c, which segfaulted release builds) was originally found by exactly that combination. I will write that up with the measurements rather than expand the scope of this PR.