Repository navigation
Py_CLEAR/Py_SETREF() regression on the main branch with C++: SEGV with simple numpy program #157649
Description
Bug report
Bug description:
The program that triggers the crash:
import numpy
for _ in range(200):
numpy.prod(2, dtype=numpy.intp)Reproducer at https://github.com/lucascolley/cpython-np-prod-repro. This produces the SEGV on CPython main branch, but passes on the 3.15 branch.
❯ export BUILD_EDITABLE_PYTHON=false
❯ pixi run repro
✨ Pixi task (repro): python repro.py
Python: 3.16.0a0 (heads/master-dirty:d8a1072491, Sep 16 2026, 22:19:03) [Clang 21.1.8 ]
AddressSanitizer:DEADLYSIGNAL
=================================================================
==54023==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00010639c214 bp 0x00016afdf230 sp 0x00016afdf1d0 T0)
==54023==The signal is caused by a WRITE memory access.
==54023==HINT: address points to the zero page.
#0 0x00010639c214 in _PyGCHead_SET_NEXT pycore_gc.h:151
#1 0x00010639c214 in _PyObject_GC_UNTRACK pycore_gc.h:264
#2 0x00010639c214 in type_dealloc typeobject.c:6898
#3 0x000106338e40 in _Py_Dealloc object.c:3333
#4 0x00010e880a34 in PyUFunc_GenericReduction+0x119c (_multiarray_umath.cpython-316-darwin.so:arm64+0x704a34)
#5 0x000106237858 in _PyVectorcall_Call call.c:273
#6 0x00010652b77c in _PyEval_EvalFrameDefault generated_cases.c.h:2831
#7 0x0001065153bc in _PyEval_EvalFrame pycore_ceval.h:122
#8 0x0001065153bc in _PyEval_Vector ceval.c:2176
#9 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
#10 0x000106237a78 in PyObject_Vectorcall call.c:327
#11 0x00010e54bd1c in dispatcher_vectorcall+0xc70 (_multiarray_umath.cpython-316-darwin.so:arm64+0x3cfd1c)
#12 0x000106237a78 in _PyObject_VectorcallTstate pycore_call.h:144
#13 0x000106237a78 in PyObject_Vectorcall call.c:327
#14 0x000106515938 in _Py_VectorCall_StackRefSteal ceval.c:728
#15 0x000106529170 in _PyEval_EvalFrameDefault generated_cases.c.h:3686
#16 0x000106514db0 in _PyEval_EvalFrame pycore_ceval.h:122
#17 0x000106514db0 in _PyEval_Vector ceval.c:2176
#18 0x000106514db0 in PyEval_EvalCode ceval.c:681
#19 0x00010665ba10 in run_mod pythonrun.c:1509
#20 0x00010665633c in _PyRun_File pythonrun.c:1332
#21 0x00010665633c in _PyRun_SimpleFile pythonrun.c:544
#22 0x000106655998 in _PyRun_AnyFile pythonrun.c:92
#23 0x0001066bd0b0 in pymain_run_file_obj main.c:478
#24 0x0001066bd0b0 in pymain_run_file main.c:494
#25 0x0001066bbe68 in pymain_run_python main.c:804
#26 0x0001066bbe68 in Py_RunMain main.c:891
#27 0x0001066bc830 in pymain_main main.c:921
#28 0x0001066bc9e0 in Py_BytesMain main.c:945
#29 0x000189febdfc in start+0x1b4c (dyld:arm64e+0x1fdfc)
==54023==Register values:
x[0] = 0x00006190000dde88 x[1] = 0x000000010639c054 x[2] = 0x000000016afde9d0 x[3] = 0x000061600009a870
x[4] = 0x0000000000000218 x[5] = 0x0000000000000218 x[6] = 0x0000000169fe4000 x[7] = 0x0000000000000001
x[8] = 0x00006190000dde80 x[9] = 0x0000000000000000 x[10] = 0x0000000000000000 x[11] = 0x00000fffffffffff
x[12] = 0x0000000000000000 x[13] = 0xffffffffffffffff x[14] = 0x0000000000000000 x[15] = 0x00007fffffffffff
x[16] = 0x0000000000000028 x[17] = 0x0000604000000410 x[18] = 0x0000000000000000 x[19] = 0x00006190000dde90
x[20] = 0x0000604000000438 x[21] = 0x0000000080805980 x[22] = 0x0000008000020000 x[23] = 0x0000000106c42940
x[24] = 0x0000008000020000 x[25] = 0x0000000000000000 x[26] = 0x000000016afdfb20 x[27] = 0x0000000000000000
x[28] = 0x00006110001f42e4 fp = 0x000000016afdf230 lr = 0x000000010639c1d8 sp = 0x000000016afdf1d0
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV pycore_gc.h:151 in _PyGCHead_SET_NEXT
==54023==ABORTING@ngoldbaum perhaps you can reproduce?
[workspace]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["osx-arm64"]
preview = ["pixi-build"]
requires-pixi = ">=0.74.0"
[tasks]
repro = "python repro.py"
[dependencies]
numpy = {
git = "https://github.com/numpy/numpy",
package = {
host-dependencies = {
python = {
flags = ["asan"],
# passing with 3.15 branch
# branch = "3.15",
# failing with main
branch = "main",
git = "https://github.com/python/cpython",
subdirectory = "Tools/pixi-packages",
},
meson-python = "*",
cython = "*",
},
build = {
backend.name = "pixi-build-python",
config.compilers = ["c", "cxx"],
},
},
}CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-157649: Restore safe C++ refcount macro temporaries #157718
- gh-157649: Fix Py_CLEAR()/Py_SETREF() on C++ #158070
- gh-157649: Merge test_cppext into test_cext #158176
- gh-157649: Check for reference leak in test_cext #158227
- [3.15] gh-157649: Fix Py_CLEAR()/Py_SETREF() on C++ #159143
- [3.15] gh-157649: Merge test_cppext into test_cext (#158176) #159264
- [3.14] gh-157649: Merge test_cppext into test_cext (#158176) (#159264) #159276
- [3.15] gh-157649: Check for reference leak in test_cext #159296
- [3.14] gh-157649: Check for reference leak in test_cext (#159296) #159314
- [3.14] gh-157649: Skip limited C API tests on Free Threading #159371
- [3.15] gh-157649: Use more warmups in test_cext to check for refleaks #159374
- [3.14] gh-157649: Use more warmups in test_cext to check for refleaks #159375
- gh-157649: Use more warmups in test_cext to check for refleaks #159378
Activity
lucascolley commented on Sep 16, 2026
Update: bisected to bba99f2 gh-157267
ngoldbaum commented on Sep 16, 2026
ping @vstinner
lucascolley commented on Sep 16, 2026
runtime packages:
DetailsCPython build-time deps info:
Detailsngoldbaum commented on Sep 17, 2026
Here's an AI-generated C++ example that triggers the same issue NumPy is triggering:
DetailsThe decltype of an lvalue expression like slots[0] is a reference type rather than an object type. That means with the new semantics, the "old value" temporary in Py_SETREF/Py_XSETREF/Py_CLEAR ends up as an alias of the slot being overwritten.
StanFromIreland commented on Sep 17, 2026
As I noted there, that's not possible as the commit was not backported to 3.15. It's only on main.
Sorry, I misread the issue, 3.15 is not failing if I understand correctly.
lucascolley commented on Sep 17, 2026
any ideas what a fix might look like?
If anyone wants to test a potential fix, you can edit https://github.com/lucascolley/cpython-np-prod-repro/blob/main/pixi.toml#L20-L21 to point to a local path, or a revision on a git fork
vstinner commented on Sep 17, 2026
I built and installed Python in /opt/py316 with:
I built @ngoldbaum reproducer with:
The reproducer does crash without ASAN:
vstinner commented on Sep 17, 2026
When running g++ -E, the C++ code:
becomes:
I can reproduce the crash with this simple C++ code which doesn't use Python:
The following C++ program doesn't crash, it uses Python 3.15 Py_CLEAR() implementation:
lucascolley commented on Sep 17, 2026
(indeed, this diff reproduces the SIGSEGV (albeit without the memory address traceback))
29 remaining items
gh-157649: Check for reference leak in test_cext (#158227)
gh-157649: Fix Py_CLEAR()/Py_SETREF() on C++ (#158070)
vstinner commented on Sep 28, 2026
Thanks for your bug report. The issue should now be fixed in the main branch, and tests have been added to check for non regression.
lucascolley commented on Sep 28, 2026
thank you @vstinner and all for the extensive work here! Glad to have helped find this :)
vstinner commented on Oct 4, 2026
If there are still C++ experts around, I would like your opinion about gh-158810 issue (Py_ARRAY_LENGTH). I'm not sure how to fix it.
[3.15] gh-157649: Fix Py_CLEAR()/Py_SETREF() on C++ (#159143)
[3.15] gh-157649: Merge test_cppext into test_cext (#158176) (#159264)
[3.14] gh-157649: Merge test_cppext into test_cext (#158176) (#159264) (
[3.15] gh-157649: Check for reference leak in test_cext (#159296)
[3.14] gh-157649: Check for reference leak in test_cext (#159296) (#1…
[3.14] gh-157649: Skip limited C API tests on Free Threading (#159371)
[3.15] gh-157649: Use more warmups in test_cext to check for refleaks (…
[3.14] gh-157649: Use more warmups in test_cext to check for refleaks (…
gh-157649: Use more warmups in test_cext to check for refleaks (#159378)