Repository navigation
test_manual_unwind_respects_frame_pointers fails on x86_64 Fedora Linux with enabled JIT (yes-off) #144741
Description
Bug report
Bug description:
We build Python on x86_64 and aarch64 with --enable-experimental-jit=yes-off . The newly added test passes for aarch64 but doesn't for x86_64, with the following traceback.
For completeness, the CFLAGS we build with are:
CFLAGS=' -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer 'I am unable to reproduce this outside of the rpm build context yet, but I'm happy to test any ideas.
Full traceback:
Re-running test_frame_pointer_unwind in verbose mode (matching: test_manual_unwind_respects_frame_pointers)
test_manual_unwind_respects_frame_pointers (test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers) ... #00 0x7f8763378fda -> other
#01 0x7f87632eeac7 -> other
#02 0x7f87632efb77 -> other
#03 0x7f87634ddd3c -> other
#04 0x7f87634e6604 -> other
#05 0x7f87634da121 -> other
#06 0x7f876351552d -> other
#07 0x7f87632eff85 -> other
#08 0x7f87632eeac7 -> other
#09 0x7f87632efb77 -> other
#10 0x7f8763852186 -> other
#11 0x7f87634de1f1 -> other
#12 0x7f87634e8cdc -> other
#13 0x7f87634da121 -> other
#14 0x7f876351552d -> other
#15 0x7f87632eff85 -> other
#16 0x7f87632eeac7 -> other
#17 0x7f87632efb77 -> other
#18 0x7f8763852186 -> other
#19 0x7f87634de1f1 -> other
#20 0x7f87634e8cdc -> other
#21 0x7f87634da121 -> other
#22 0x7f876351552d -> other
#23 0x7f87632eff85 -> other
#24 0x7f87632eeac7 -> other
#25 0x7f87632efb77 -> other
#26 0x7f8763852186 -> other
#27 0x7f87634de1f1 -> other
#28 0x7f87634e8cdc -> other
#29 0x7f87634da121 -> other
#30 0x7f876351552d -> other
#31 0x7f87632eff85 -> other
#32 0x7f87632eeac7 -> other
#33 0x7f87632efb77 -> other
#34 0x7f8763852186 -> other
#35 0x7f87634de1f1 -> other
#36 0x7f87634e8cdc -> other
#37 0x7f87634da121 -> other
#38 0x7f876351552d -> other
#39 0x7f87632eff85 -> other
#40 0x7f87632eeac7 -> other
#41 0x7f87632efb77 -> other
#42 0x7f8763852186 -> other
#43 0x7f87634de1f1 -> other
#44 0x7f87634e8cdc -> other
#45 0x7f87634da121 -> other
#46 0x7f876351552d -> other
#47 0x7f87632eff85 -> other
#48 0x7f87632eeac7 -> other
#49 0x7f87632efb77 -> other
#50 0x7f8763852186 -> other
#51 0x7f87634de1f1 -> other
#52 0x7f87634e8cdc -> other
#53 0x7f87634da121 -> other
#54 0x7f876351552d -> other
#55 0x7f87632eff85 -> other
#56 0x7f87632eeac7 -> other
#57 0x7f87632efb77 -> other
#58 0x7f8763852186 -> other
#59 0x7f87634de1f1 -> other
#60 0x7f87634e8cdc -> other
#61 0x7f87634da121 -> other
#62 0x7f876351552d -> other
#63 0x7f87632eff85 -> other
#64 0x7f87632eeac7 -> other
#65 0x7f87632efb77 -> other
#66 0x7f8763852186 -> other
#67 0x7f87634de1f1 -> other
#68 0x7f87634e8cdc -> other
#69 0x7f87634da121 -> other
#70 0x7f876351552d -> other
#71 0x7f87632eff85 -> other
#72 0x7f87632eeac7 -> other
#73 0x7f87632efb77 -> other
#74 0x7f8763852186 -> other
#75 0x7f87634de1f1 -> other
#76 0x7f87634e8cdc -> other
#77 0x7f87634da121 -> other
#78 0x7f876351552d -> other
#79 0x7f87632eff85 -> other
#80 0x7f87632eeac7 -> other
#81 0x7f87632efb77 -> other
#82 0x7f8763852186 -> other
#83 0x7f8763378e09 -> other
#84 0x7f87632eeac7 -> other
#85 0x7f87632efb77 -> other
#86 0x7f87634ddd3c -> other
#87 0x7f87634e6604 -> other
#88 0x7f87634da121 -> other
#89 0x7f876351552d -> other
#90 0x7f87634dd758 -> other
#91 0x7f876374d7ed -> other
#92 0x7f876374dc58 -> other
#93 0x7f876374d40e -> other
#94 0x7f876374b98d -> other
#95 0x7f876378b0fd -> other
#96 0x7f876378c121 -> other
#97 0x7f876378c2f3 -> other
#98 0x7f876378c3c9 -> other
#99 0x7f876378c491 -> other
#100 0x55f0d60984bd -> python
#101 0x7f8763008681 -> other
#102 0x7f8763008798 -> other
#103 0x55f0d60983d5 -> python
{"length": 104, "python_frames": 2, "jit_frames": 0, "other_frames": 102, "jit_backend": "jit"}
JIT ranges:
0x7fb224624000-0x7fb224647000
#00 0x7fb233378fda -> other
#01 0x7fb2332eeac7 -> other
#02 0x7fb2332efb77 -> other
#03 0x7fb2334ddd3c -> other
#04 0x7fb2334e6604 -> other
#05 0x7fb2334da121 -> other
#06 0x7fb23351552d -> other
#07 0x7fb2332eff85 -> other
#08 0x7fb2332eeac7 -> other
#09 0x7fb2332efb77 -> other
#10 0x7fb233852186 -> other
#11 0x7fb2334de1f1 -> other
#12 0x7fb22462f00f -> jit
{"length": 13, "python_frames": 0, "jit_frames": 1, "other_frames": 12, "jit_backend": "jit"}
test_manual_unwind_respects_frame_pointers (test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers) (env={'PYTHON_JIT': '1'}) ... FAIL
======================================================================
FAIL: test_manual_unwind_respects_frame_pointers (test.test_frame_pointer_unwind.FramePointerUnwindTests.test_manual_unwind_respects_frame_pointers) (env={'PYTHON_JIT': '1'})
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/python3.15-3.15.0_a6-build/Python-3.15.0a6/Lib/test/test_frame_pointer_unwind.py", line 182, in test_manual_unwind_respects_frame_pointers
self.assertGreater(
~~~~~~~~~~~~~~~~~~^
python_frames,
^^^^^^^^^^^^^^
0,
^^
f"expected to find Python frames on {self.machine} with env {env}",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
AssertionError: 0 not greater than 0 : expected to find Python frames on x86_64 with env {'PYTHON_JIT': '1'}
CPython versions tested on:
3.15
Operating systems tested on:
Linux
Activity
vstinner commented on Mar 4, 2026
I reproduced the failure on Fedora Rawhide with the following commands:
Fedora Rawhide: gcc (GCC) 16.0.1 20260209 (Red Hat 16.0.1-0).
vstinner commented on Mar 4, 2026
Aha, I can also reproduce the failure on Fedora 43: gcc (GCC) 15.2.1 20260123 (Red Hat 15.2.1-7).
vstinner commented on Mar 4, 2026
I reduced the reproducer to:
vstinner commented on Mar 4, 2026
script.py:
Python built with --enable-shared
Output with JIT disabled (truncated): python_frames=2
$ PYTHON_JIT=0 LD_LIBRARY_PATH=$PWD ./python script.py #00 0x7effb2cfc1d6 -> other #01 0x7effb2ca2e2e -> other (...) #100 0x7effb2fa50b5 -> other #101 0x7effb2fa5135 -> other #102 0x40046f -> python #103 0x7effb2a105b5 -> other #104 0x7effb2a10668 -> other #105 0x4003a5 -> python {"length": 106, "python_frames": 2, "jit_frames": 0, "other_frames": 104, "jit_backend": "jit"}Output with JIT enabled: python_frames=0
$ PYTHON_JIT=1 LD_LIBRARY_PATH=$PWD ./python script.py JIT ranges: 0x7eff619cf000-0x7eff619f2000 #00 0x7eff706fc1d6 -> other #01 0x7eff706a2e2e -> other #02 0x7eff706a2f2e -> other #03 0x7eff707edbd6 -> other #04 0x7eff707f563c -> other #05 0x7eff7080f2a2 -> other #06 0x7eff7080f477 -> other #07 0x7eff706a2ad1 -> other #08 0x7eff706a2e2e -> other #09 0x7eff706a2f2e -> other #10 0x7eff70a24446 -> other #11 0x7eff707ede80 -> other #12 0x7eff619d9e9b -> jit {"length": 13, "python_frames": 0, "jit_frames": 1, "other_frames": 12, "jit_backend": "jit"}Python built without --enable-shared
Output with JIT disabled (truncated): python_frames=104
$ PYTHON_JIT=0 LD_LIBRARY_PATH=$PWD ./python script.py #00 0x4e9e03 -> python #01 0x495fc5 -> python (...) #101 0x774fb7 -> python #102 0x401cef -> python #103 0x7f76331675b5 -> other #104 0x7f7633167668 -> other #105 0x401c25 -> python {"length": 106, "python_frames": 104, "jit_frames": 0, "other_frames": 2, "jit_backend": "jit"}Output with JIT enabled: python_frames=12
$ PYTHON_JIT=1 LD_LIBRARY_PATH=$PWD ./python script.py JIT ranges: 0x7f5b44c50000-0x7f5b44c73000 #00 0x4e9e03 -> python #01 0x495fc5 -> python #02 0x49608d -> python #03 0x5cbe18 -> python #04 0x5d2f17 -> python #05 0x5eb2ae -> python #06 0x5eb460 -> python #07 0x495cb1 -> python #08 0x495fc5 -> python #09 0x49608d -> python #10 0x7ee49e -> python #11 0x5cc0b0 -> python #12 0x7f5b44c5ae9b -> jit {"length": 13, "python_frames": 12, "jit_frames": 1, "other_frames": 0, "jit_backend": "jit"}pythongh-144741: Fix test_frame_pointer_unwind for libpython
pythongh-144741: Fix test_frame_pointer_unwind for libpython
vstinner commented on Mar 4, 2026
I wrote #145499 to fix this issue.
gh-144741: Fix test_frame_pointer_unwind for libpython (#145499)
vstinner commented on Mar 4, 2026
Fixed by the change a003923. Thanks for your bug report @befeleme.
ferdnyc commented on Mar 10, 2026
Unfortunately, even after #145499 I'm seeing test_frame_pointer_unwind segfaults on Fedora 43 x86_64 after building Python using system GCC 15.2.1 (wrapped with automatic ccache), in an out-of-tree builddir (in case that's significant) configured with --enable-shared. Exact commands used:
Output for failing test:
hroncok commented on Mar 10, 2026
I was about to say that 3.15.0a7 indeed fixed this for us. Will try on Fedora 43 as well.
ferdnyc commented on Mar 10, 2026
Running the individual test through gdb, the segfault seems to happen in the new _testinternalcapi.c function manual_unwind_from_fp, when it gets passed a very suspicious frame_pointer value:
(return_addr is set to frame_pointer[1] a few lines earlier, but given that it appears to be no sort of pointer..)
Here's the start of the backtrace, up to the point it gets ugly and redundant. Not that it's very useful with the arguments to manual_frame_pointer_unwind optimized out:
(Is nargsf supposed to be an argument count? Because if so, seems like something is already way wrong.)
ferdnyc commented on Mar 10, 2026
I'm running on x86_64, as I said, building without any omit-frame-pointer flags in the CFLAGS. As a result, when the test's setUp method runs _frame_pointers_expected("x86_64"), it gets back False, as expected. (Confirmed by manually defining and executing the function in the REPL.)
It then executes this:
...Which is where it segfaults.
I assume if I was building with -fno-omit-frame-pointer (like the Fedora official builds are, according to @befeleme's OP) then it wouldn't be an issue, but my build doesn't have that compile flag set.
ferdnyc commented on Mar 11, 2026
Huh. Stepping through with gdb, I'm inside a manual_unwind_from_fp() call that was passed a valid ("valid-looking") frame_pointer=0x7fffffffb6f0. But here's what I get if I pick that value apart roughly the way the code does:
So, that's where the crazy-looking frame_pointer=0x8000000000000000 is coming from; it's when the code gets to the second iteration of this loop, because next_fp is apparently neither NULL, nor less than min_frame_pointer_addr, nor a valid frame pointer...
Methinks something is not the size/shape we think it is. (The return type from __builtin_frame_address() is void*, after all.)
cpython/Modules/_testinternalcapi.c
Lines 345 to 384 in 5197ecb
vstinner commented on Mar 11, 2026
Please open a new issue for your crash.
vstinner commented on Mar 11, 2026
Thanks for the confirmation, I close the issue.
Update python315 to version 3.15.0~a7 / rev 8 via SR 1341615
mcepl commented on Apr 8, 2026
What’s the relationship between this and #126910 ?
vstinner commented on Apr 8, 2026
This issue is about a failure of test_frame_pointer_unwind.
test_frame_pointer_unwind, _testinernalcapi.manual_frame_pointer_unwind() and _testinternalcapi.classify_stack_addresses() were added by the issue GH-126910 (commit 66055d0).
pythongh-144741: Fix test_frame_pointer_unwind for libpython (python#…