FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

GH-140479: Update JIT builds to use LLVM 21 by savannahostrowski · Pull Request #140973 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

GH-140479: Update JIT builds to use LLVM 21 - #140973

Merged
savannahostrowski merged 15 commits into
python:mainfrom
savannahostrowski:llvm-21
Nov 12, 2025
ConversationCommits (15)ChecksFiles changed
Merged

GH-140479: Update JIT builds to use LLVM 21#140973
savannahostrowski merged 15 commits into
python:mainfrom
savannahostrowski:llvm-21

Conversation

savannahostrowski commented Nov 4, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Well, here's a fun treat...looks like the logic to fetch the LLVM binary as a release artifact from https://github.com/python/cpython-bin-deps/releases/tag/llvm-21.1.4.0 never actually worked 😓. CI only passed for Windows on #140329 because LLVM 20 is included in the base image we use for Windows builds (example: https://github.com/actions/partner-runner-images/blob/main/images/arm-windows-11-image.md#language-and-runtime).

As such, we should merge #141002 before this one (Windows CI will fail otherwise!).

savannahostrowski added the build The build process and cross-build label Nov 4, 2025
savannahostrowski added the dependencies Pull requests that update a dependency file label Nov 4, 2025
savannahostrowski added the infra CI, GitHub Actions, buildbots, Dependabot, etc. label Nov 4, 2025
bedevere-app Bot mentioned this pull request Nov 4, 2025
Closed
savannahostrowski mentioned this pull request Nov 4, 2025
Merged
savannahostrowski commented Nov 4, 2025
View reviewed changes
Comment thread Tools/jit/README.md

diegorusso commented Nov 4, 2025

Copy link
Copy Markdown
Contributor

Well, here's a fun treat...looks like the logic to fetch the LLVM binary as a release artifact from https://github.com/python/cpython-bin-deps/releases/tag/llvm-21.1.4.0 never actually worked 😓. CI only passed for Windows on #140329 because LLVM 20 is included in the base image we use for Windows builds (see https://github.com/actions/partner-runner-images/blob/main/images/arm-windows-11-image.md#language-and-runtime).

This PR patches up the logic for extracting the tarball in get_externals.py and then...just bumps the version to 21.

What about the Windows x86_64 image? Does it have llvm 20 as well?

Also, in this PR we do 2 things: fix some logic and bump to 21. Wouldn't it be better to do 2 PRs? In case we need to revert to llvm 20 (for any reason) we will revert to a working fetching for version 20.

savannahostrowski mentioned this pull request Nov 4, 2025
Merged

savannahostrowski commented Nov 4, 2025

Copy link
Copy Markdown
Member Author

@diegorusso Same deal with x86_64 - see https://github.com/actions/runner-images/blob/win22/20251021.76/images/windows/Windows2022-Readme.md#language-and-runtime

Sure - see #141002

diegorusso approved these changes Nov 4, 2025
View reviewed changes

diegorusso left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Provided #141002 gets merged before, this can go in as well. Thanks for splitting them up.

Fidget-Spinner approved these changes Nov 7, 2025
View reviewed changes

Fidget-Spinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Looks simple enough, let's do this!

pablogsal commented Nov 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

I have tried to rebase https://github.com/pablogsal/cpython-unwind to check the unwinding. Looks like the function to test now needs to be more complicated beause the JIT refuses to call functions if there are branches now and requires tons of iterations to start jitting. I would love if someone can take a look at run.py. With the code we can validate all unwinding methods

Fidget-Spinner commented Nov 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

The main branch's JIT doesn't seem to want to compile the example due to trace stack underflow. So I reduced the side exit count on my tracing JIT example and tried. Seems like LLVM 21 doesn't fix the problem :(

PYTHON_JIT=0
-----------
Testing various unwinding methods on the same deep call stack...

[GNU backtrace]
Total frames: 27
Showing first 10 frames:
  # 0  /home/ken/Documents/GitHub/cpython-unwind/stackunwind.cpython-315-x86_64-linux-gnu.so(+0x280a) [0x78674f7a580a]
  # 1  ../cpython/python(PyObject_Vectorcall+0x33) [0x58c0e08d2303]
  # 2  ../cpython/python(_PyEval_EvalFrameDefault+0xf7a1) [0x58c0e086b141]
  # 3  ../cpython/python(+0x2860aa) [0x58c0e0a430aa]
  # 4  ../cpython/python(PyObject_Vectorcall+0x33) [0x58c0e08d2303]
  # 5  ../cpython/python(_PyEval_EvalFrameDefault+0x3fb3) [0x58c0e085f953]
  # 6  ../cpython/python(+0x2860aa) [0x58c0e0a430aa]
  # 7  ../cpython/python(PyObject_Vectorcall+0x33) [0x58c0e08d2303]
  # 8  ../cpython/python(_PyEval_EvalFrameDefault+0x3fb3) [0x58c0e085f953]
  # 9  ../cpython/python(+0x2860aa) [0x58c0e0a430aa]
  ... (17 more frames)

[libunwind]
Total frames: 26
Showing first 10 frames:
  # 0  PyObject_Vectorcall+0x33
  # 1  _PyEval_EvalFrameDefault+0xf7a1
  # 2  _PyEval_Vector+0x21a
  # 3  PyObject_Vectorcall+0x33
  # 4  _PyEval_EvalFrameDefault+0x3fb3
  # 5  _PyEval_Vector+0x21a
  # 6  PyObject_Vectorcall+0x33
  # 7  _PyEval_EvalFrameDefault+0x3fb3
  # 8  _PyEval_Vector+0x21a
  # 9  PyObject_Vectorcall+0x33
  ... (16 more frames)

[libdw (DWARF)]
Total frames: 26
Showing first 10 frames:
  # 0  PyObject_Vectorcall
  # 1  _PyEval_EvalFrameDefault
  # 2  _PyEval_Vector
  # 3  PyObject_Vectorcall
  # 4  _PyEval_EvalFrameDefault
  # 5  _PyEval_Vector
  # 6  PyObject_Vectorcall
  # 7  _PyEval_EvalFrameDefault
  # 8  _PyEval_Vector
  # 9  PyObject_Vectorcall
  ... (16 more frames)

[Manual frame pointers]
Total frames: 4
Showing first 4 frames:
  # 0  0x58c0e08d2303
  # 1  0x2
  # 2  0x58c0e0dbb7e0
  # 3  0x2


PYTHON_JIT=1
------------
[GNU backtrace]
Total frames: 6
Showing first 6 frames:
  # 0  /home/ken/Documents/GitHub/cpython-unwind/stackunwind.cpython-315-x86_64-linux-gnu.so(+0x280a) [0x7a0a357d080a]
  # 1  ../cpython/python(PyObject_Vectorcall+0x33) [0x57385e3d3303]
  # 2  ../cpython/python(_PyEval_EvalFrameDefault+0xf7a1) [0x57385e36c141]
  # 3  ../cpython/python(+0x2860aa) [0x57385e5440aa]
  # 4  ../cpython/python(PyObject_Vectorcall+0x33) [0x57385e3d3303]
  # 5  [0x7a0a3398d4fb]

[libunwind]
Total frames: 5
Showing first 5 frames:
  # 0  PyObject_Vectorcall+0x33
  # 1  _PyEval_EvalFrameDefault+0xf7a1
  # 2  _PyEval_Vector+0x21a
  # 3  PyObject_Vectorcall+0x33
  # 4  <unknown>

[libdw (DWARF)]
Total frames: 5
Showing first 5 frames:
  # 0  PyObject_Vectorcall
  # 1  _PyEval_EvalFrameDefault
  # 2  _PyEval_Vector
  # 3  PyObject_Vectorcall
  # 4  <unknown>

[Manual frame pointers]
Total frames: 4
Showing first 4 frames:
  # 0  0x57385e3d3303
  # 1  0x(nil)
  # 2  0x57385e8bc7e0
  # 3  0x(nil)

Notice the missing frames!

However, I have made a very interesting discovery --- compiling the interpreter with a recent enough clang version (tested with clang-20/21) to the JIT restores the frames!!!

[GNU backtrace]
Total frames: 29
Showing first 10 frames:
  # 0  /home/ken/Documents/GitHub/cpython-unwind/stackunwind.cpython-315-x86_64-linux-gnu.so(+0x280a) [0x797cf3f7c80a]
  # 1  ../cpython/python(PyObject_Vectorcall+0x43) [0x64be6334e693]
  # 2  ../cpython/python(_PyEval_EvalFrameDefault+0x5d3e) [0x64be6347f63e]
  # 3  ../cpython/python(+0x217706) [0x64be63479706]
  # 4  ../cpython/python(PyObject_Vectorcall+0x43) [0x64be6334e693]
  # 5  ../cpython/python(_PyEval_EvalFrameDefault+0x3768) [0x64be6347d068]
  # 6  ../cpython/python(+0x217706) [0x64be63479706]
  # 7  ../cpython/python(PyObject_Vectorcall+0x43) [0x64be6334e693]
  # 8  ../cpython/python(_PyEval_EvalFrameDefault+0x3768) [0x64be6347d068]
  # 9  ../cpython/python(+0x217706) [0x64be63479706]
  ... (19 more frames)

[libunwind]
Total frames: 28
Showing first 10 frames:
  # 0  PyObject_Vectorcall+0x43
  # 1  _PyEval_EvalFrameDefault+0x5d3e
  # 2  _PyEval_Vector+0x266
  # 3  PyObject_Vectorcall+0x43
  # 4  _PyEval_EvalFrameDefault+0x3768
  # 5  _PyEval_Vector+0x266
  # 6  PyObject_Vectorcall+0x43
  # 7  _PyEval_EvalFrameDefault+0x3768
  # 8  _PyEval_Vector+0x266
  # 9  PyObject_Vectorcall+0x43
  ... (18 more frames)

[libdw (DWARF)]
Total frames: 28
Showing first 10 frames:
  # 0  PyObject_Vectorcall
  # 1  _PyEval_EvalFrameDefault
  # 2  _PyEval_Vector
  # 3  PyObject_Vectorcall
  # 4  _PyEval_EvalFrameDefault
  # 5  _PyEval_Vector
  # 6  PyObject_Vectorcall
  # 7  _PyEval_EvalFrameDefault
  # 8  _PyEval_Vector
  # 9  PyObject_Vectorcall
  ... (18 more frames)

[Manual frame pointers]
Total frames: 16
Showing first 10 frames:
  # 0  0x64be6334e693
  # 1  0x2
  # 2  0x64be63479706
  # 3  0x2
  # 4  0x64be63479706
  # 5  0x2
  # 6  0x64be63479706
  # 7  0x(nil)
  # 8  0x64be63479706
  # 9  0x(nil)
  ... (6 more frames)

So the solution to the stack unwinding seems to be to mandate compiilng CPython with the same clang as the JIT!

pablogsal commented Nov 9, 2025

Copy link
Copy Markdown
Member

So the solution to the stack unwinding seems to be to mandate compiilng CPython with the same clang as the JIT!

We need to understand what the hell it's going on here because otherwise the risk it's that this breaks randomly in the future

Fidget-Spinner commented Nov 9, 2025

Copy link
Copy Markdown
Member

Another data point: old GCC (11) compiled with frame pointers with JIT also allows us to see the frames.

brandtbucher commented Nov 10, 2025

Copy link
Copy Markdown
Member

Just to be clear, merely upgrading to LLVM 21 isn’t sufficient to get unwinding. We also need to do the reserved frame pointer thing, which isn’t in this branch.

brandtbucher commented Nov 10, 2025

Copy link
Copy Markdown
Member

That can come after.

savannahostrowski commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

Yeah, was chatting with Brandt earlier this weekend about this and will probably go investigate the unwinding piece after this gets merged.

Fidget-Spinner commented Nov 10, 2025

Copy link
Copy Markdown
Member

I think it would be good to merge this soon. The meta benchmarking runners don't have JIT results this week because they don't have LLVM 20. It would be a little bit of churn to request they upgrade twice (once for LLVM 20, once for LLVM 21), so I was thinking if I could ask them nicely to just upgrade to LLVM 21 once :).

savannahostrowski commented Nov 12, 2025

Copy link
Copy Markdown
Member Author

Discussed in our weekly sync, merging 🚀 !

savannahostrowski enabled auto-merge (squash) November 12, 2025 17:13
savannahostrowski merged commit d162c42 into python:main Nov 12, 2025
109 of 111 checks passed
chris-eibl mentioned this pull request Nov 30, 2025
Closed
savannahostrowski deleted the llvm-21 branch December 4, 2025 22:59
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
pythonGH-140479: Update JIT builds to use LLVM 21 (python#140973)
f0d1af7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

diegorusso diegorusso approved these changes
Fidget-Spinner Fidget-Spinner approved these changes
brandtbucher Awaiting requested review from brandtbucher brandtbucher is a code owner
ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner
hugovk Awaiting requested review from hugovk hugovk is a code owner
AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Assignees

No one assigned

Labels

build The build process and cross-build dependencies Pull requests that update a dependency file infra CI, GitHub Actions, buildbots, Dependabot, etc. topic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

Back | FazBrowse Home | New Git URL