GH-139922: Tail calling for MSVC (VS 2026) by chris-eibl · Pull Request #143068 · python/cpython · GitHub

So much has happened on main that I started fresh over and sqash merged the needed things from #139962 onto latest main (mostly because I wanted to use it for my own experiments on main again).

Based on @markshannon's work in #142228 I've factored out _Py_VectorCallInstrumentation_StackRefSteal, because then @Fidget-Spinner's #138115 is not needed in this PR and can be done in a follow-up. An alternative would be to keep it and pass tstate into the Py_*_StackRefSteal functions.

Issue: Tail calling interpreter for MSVC #139922

…l-call-new

…tOptionalItem

In the future, please communicate if you want to supercede a PR for other contributors. It's often considered not good etiquette to take over another person's PR unless they explicitly allow it. However, I'm more than happy to let you take over the PR in this case.

It needed lots of trial and error to come up with this unintuive minimal change needed to convince MSVC that the local variable passed by reference into result here does not escape the tail call.

Using it as an return value in PyMapping_GetOptionalItem2 is more explicit, but might have other drawbacks?

In the future, please communicate if you want to supercede a PR for other contributors. It's often considered not good etiquette to take over another person's PR unless they explicitly allow it. However, I'm more than happy to let you take over the PR in this case.

Sorry, wasn't ment as superseding, more as a follow up and maybe just a draft for new discussions. Thought about you're gonna take whatever deems appropriate back into your original PR ...

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

Co-authored-by is the way:

https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors

If this here is going to fly, I'd like to include you and Brandt as authors for the commits, but don't know how atm ...

Oh I'm happy to be superseded. If you'd like to attribute commit history though, you can just push a trivial commit with the co-author info like Hugo pointed out, and when I squash and merge I'll use that.

Co-Authored-By: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Co-Authored-By: Brandt Bucher <brandt@python.org>

Try pulling main in, I fixed a JIT bug there that should turn the JIT CI green again.

Co-authored-by is the way:

Done

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

I could not add this one to batch, so have to do it separately Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

https://github.com/python/cpython/actions/runs/20437258430/job/58722460806 failed inbetween
test_extractall_none_gid (test.test_tarfile.NoneInfoExtractTests_Tar.test_extractall_none_gid) ... Windows fatal exception: access violation
(but got green when rerunning the failed test) even though pulling in main.

I've found this: #143073

Hopefully, this will fix it ...

try to fix "Native Windows MSVC (release)"

Just ignore the JIT problems for now. The new JIT executor management has a known bug #143026

add --no-progress to choco

🤖 New build scheduled with the buildbot fleet by @Fidget-Spinner for commit da1adaf 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F143068%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

Benchmark results are within noise for the default interpreter https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251222-3.15.0a3%2B-da1adaf/bm-20251222-vultr-x86_64-chris%252deibl-msvc_tail_call_new-3.15.0a3%2B-da1adaf-vs-base.md

Refleak bots are green.

Once again:

This PR is the successor of one that has been up for 2 months already #139962 . We've had enough discussion there to get a sensing of what we want.
Refleak tests are green, performance results are within noise for the default (computed goto) interpreter.
This PR adds CI for the new configuration, which is also green.
I have discussed this feature with the other core devs a few months back on our bi-weekly perf meetings. Everyone was in agreement that adding scopes to MSVC is fine as it isn't that invasive.
This PR is the minimal changeset required to get TC working on MSVC.
It uses the techniques suggested by Mark in the other PR (just adding scopes and commenting on them). So I expect he should be okay with the overall design I hope. I understand he might disagree on some of the smaller changes.
It LGTM, from my review of the changes.

Therefore, I'm merging this PR.

Any follow-ups can be done on future PRs. Any discussion about the design can be done on the issue itself.

chris-eibl added 3 commits December 21, 2025 14:51

rebased + _Py_VectorCallInstrumentation_StackRefSteal

374874c

Merge commit '374874c3e6d8c4745a52b3bc22044db607b2c9e7' into msvc-tai…

e0cd0f5

…l-call-new

try to fix MSVC tail call CI

7e5555a

bedevere-app Bot mentioned this pull request Dec 22, 2025

Tail calling interpreter for MSVC #139922

Closed

chris-eibl marked this pull request as ready for review December 22, 2025 14:23

chris-eibl requested review from a team, AA-Turner, ezio-melotti, hugovk and markshannon as code owners December 22, 2025 14:23

bedevere-app Bot added the awaiting review label Dec 22, 2025

chris-eibl requested a review from Fidget-Spinner December 22, 2025 14:23

Alternative to restrict and conditional Py_NO_INLINE for PyMapping_Ge…

a75042a

…tOptionalItem

Fidget-Spinner reviewed Dec 22, 2025

View reviewed changes

Comment thread Python/bytecodes.c

Fidget-Spinner reviewed Dec 22, 2025

View reviewed changes

Comment thread Include/abstract.h Outdated

chris-eibl commented Dec 22, 2025

View reviewed changes

make _PyMapping_GetOptionalItem2 private and comment scopes

fdfd8a0

Co-Authored-By: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com> Co-Authored-By: Brandt Bucher <brandt@python.org>

Merge with main because aarch64 jit builds started failing

058889e

hugovk reviewed Dec 22, 2025

View reviewed changes

Comment thread .github/workflows/tail-call.yml

Comment thread .github/workflows/tail-call.yml Outdated

chris-eibl commented Dec 22, 2025

View reviewed changes

Comment thread .github/workflows/tail-call.yml

chris-eibl and others added 2 commits December 22, 2025 17:44

Apply suggestions from code review

c8d051b

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

Update .github/workflows/tail-call.yml

ab70524

I could not add this one to batch, so have to do it separately Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

enable "Emulated Windows Clang (release)"

5c1c240

try to fix "Native Windows MSVC (release)"

chris-eibl added 2 commits December 22, 2025 18:26

Merge remote-tracking branch 'upstream/main' into msvc-tail-call-new

a04824f

revert to cmd but keep Hugo's other changes

da1adaf

add --no-progress to choco

Fidget-Spinner added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 22, 2025

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Dec 22, 2025

Fidget-Spinner approved these changes Dec 22, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Dec 22, 2025

Fidget-Spinner merged commit be3c131 into python:main Dec 22, 2025
94 of 95 checks passed

bedevere-app Bot removed the awaiting merge label Dec 22, 2025

hugovk mentioned this pull request Dec 23, 2025

Update pre-commit with zizmor and Ruff fixes #143095

Merged

chris-eibl deleted the msvc-tail-call-new branch December 26, 2025 17:45

-              // MSVC fails during a tail call release build with loads of
-              // error C4737: Unable to perform required tail call.
-              // without using Py_NO_INLINE here, but PGO works fine.
-              #if defined(_MSC_VER) && !defined(__clang__) && _Py_TAIL_CALL_INTERP && !defined(_Py_USING_PGO)

GH-139922: Tail calling for MSVC (VS 2026) by chris-eibl · Pull Request #143068 · python/cpython · GitHub

Uh oh!

Uh oh!

GH-139922: Tail calling for MSVC (VS 2026) - #143068

GH-139922: Tail calling for MSVC (VS 2026)#143068Fidget-Spinner merged 11 commits intopython:mainpython/cpython:mainfrom chris-eibl:msvc-tail-call-newchris-eibl/cpython:msvc-tail-call-newCopy head branch name to clipboard

Conversation

chris-eibl commented Dec 22, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fidget-Spinner commented Dec 22, 2025

Uh oh!

chris-eibl Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

chris-eibl commented Dec 22, 2025

Uh oh!

hugovk commented Dec 22, 2025

Uh oh!

Fidget-Spinner commented Dec 22, 2025

Uh oh!

Fidget-Spinner commented Dec 22, 2025

Uh oh!

chris-eibl commented Dec 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chris-eibl commented Dec 22, 2025

Uh oh!

Fidget-Spinner commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Dec 22, 2025

Uh oh!

Fidget-Spinner commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fidget-Spinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

GH-139922: Tail calling for MSVC (VS 2026)#143068
Fidget-Spinner merged 11 commits into
python:mainfrom
chris-eibl:msvc-tail-call-new

chris-eibl commented Dec 22, 2025 •
edited by bedevere-app Bot
Loading

Fidget-Spinner commented Dec 22, 2025 •
edited
Loading

Fidget-Spinner commented Dec 22, 2025 •
edited
Loading