gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines by gsallam · Pull Request #109666 · python/cpython · GitHub

This pull request implements a proposal in #109587 to allow precompiled perf-trampolines. This would mitigate the costs of enabling perf-trampolines, such as the disk IO and memory overhead.

The proposal introduces two new C-API functions:

PyUnstable_PerfTrampoline_CompileCode(): Creates a new trampoline and registers it.
PyUnstable_PerfTrampoline_SetPersistAfterFork(): Controls whether to re-initialize trampolines in child processes.

These functions can be used by extension modules to initialize trampolines eagerly, after the application is "warmed up". This would make it possible to have perf-trampoline running in an always-enabled fashion.

Benefits for a forked multiprocess model:

Reduces the CPU overhead of enabling perf-trampolines.
Makes it possible to have perf-trampoline running in an always-enabled fashion.

Issue: [proposal] Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines #109587

…hem in include/cpython/ceval.h

Here too?

@pablogsal , @gpshead Would love to hear your thoughts on this (whenever you get a chance to take a look) :)

Ok I resolved the merge conflict, but I think the new MacOS failure is unrelated.

LGTM

Seems that we have some conflicts that we need to address before landing, @gsallam can you take a look?

@gsallam We still have some failures in the Windows and macOS CI. I think this is because the symbol doesn't exist in those cases:

_testinternalcapi.obj : error LNK2019: unresolved external symbol __imp_PyUnstable_PerfTrampoline_CompileCode referenced in function compile_perf_trampoline_entry [D:\a\cpython\cpython\PCbuild\_testinternalcapi.vcxproj]
D:\a\cpython\cpython\PCbuild\amd64\_testinternalcapi_d.pyd : fatal error LNK1120: 1 unresolved externals [D:\a\cpython\cpython\PCbuild\_testinternalcapi.vcxproj]
    0 Warning(s)
    2 Error(s)

Summary: As titled. To be on sync with python/cpython#109666. Reviewed By: czardoz Differential Revision: D50329585 fbshipit-source-id: d4c52143216a1d9970196ad71ef656098be50120

We still have some failing tests it seems

This approach is likely quite silly, but I don't have a Windows/Mac machine handy, so I'm kicking off these builds to test/debug.

I will look at this over the weekend on my Windows box, this trial-and-error is not an optimal way :)

I will look at this over the weekend on my Windows box, this trial-and-error is not an optimal way :)

Don't worry, I pushed a commit that should fix it

Thanks a lot @pablogsal! 😄

…gate the cost of enabling perf-trampolines (python#109666)

Summary: Straight up port of python/cpython#109666 Reviewed By: jbower-fb Differential Revision: D66118319 fbshipit-source-id: bd7d4181c16d35526f57500cebb6e16fc2df58aa

The PyUnstable functions added here are missing documentation.
@Yashp002 sent a PR to add docs: #143492

gsallam added 5 commits September 20, 2023 23:19

Update sysmodule.h

bc4203b

Update ceval.h

3c805e1

Update pycore_ceval_state.h

ccc1ec5

Update perf_trampoline.c

3098e9d

Update sysmodule.c

09d65f9

bedevere-app Bot added the awaiting review label Sep 21, 2023

bedevere-app Bot mentioned this pull request Sep 21, 2023

[proposal] Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines #109587

Closed

gsallam added 3 commits September 21, 2023 09:38

Update perf_trampoline.c

83c6359

remove whitespace in perf_trampoline.c

5d2be1c

Update stable_abi.toml

f04006b

gsallam requested a review from encukou as a code owner September 22, 2023 18:00

Update test_stable_abi_ctypes.py

1d285f3

brandtbucher reviewed Sep 22, 2023

View reviewed changes

Comment thread Misc/stable_abi.toml Outdated

gsallam added 5 commits September 22, 2023 16:09

Update stable_abi.toml

3712109

Update ceval.h

86bc6b6

Add the two API to include/cpython/ceval.h instead of include/ceval.h

9edfe53

Remove the two APIs fro include/ceval.h ceval.h and instead include t…

c41fa27

…hem in include/cpython/ceval.h

remove new line from ceval.h

5132af8

encukou removed their request for review September 25, 2023 11:50

brandtbucher reviewed Sep 25, 2023

View reviewed changes

carljm reviewed Sep 25, 2023

View reviewed changes

Comment thread Include/cpython/ceval.h Outdated

gsallam and others added 3 commits September 25, 2023 11:40

Remove changes in test_stable_abi_ctypes.py

1124bc0

Remove #ifndef Py_LIMITED_API from ceval.h

70ea0de

📜🤖 Added by blurb_it.

9b64aee

gpshead self-assigned this Oct 6, 2023

gpshead self-requested a review October 6, 2023 12:11

gpshead removed their assignment Oct 6, 2023

Merge branch 'main' of github.com:python/cpython

3bb50da

pablogsal approved these changes Oct 19, 2023

View reviewed changes

Merge remote-tracking branch 'upstream/main'

a6db587

pablogsal enabled auto-merge (squash) October 19, 2023 18:41

Fix build (possibly)

1ed3205

auto-merge was automatically disabled October 19, 2023 19:06
Head branch was pushed to by a user without write access

czardoz and others added 3 commits October 19, 2023 14:17

fix bad merge

140d314

add a space before the right parenthesis perf_trampoline.c

012282e

add a missing space before the right parenthesis sysmodule.c

62ff839

AlexWaygood mentioned this pull request Oct 26, 2023

Bedevere doesn't say "thank you" for making the requested changes anymore python/bedevere#599

Closed

czardoz added 3 commits October 26, 2023 18:44

move definitions to the right header file

b3c42d1

Merge remote-tracking branch 'upstream/main'

6c3fe05

gate PyUnstable_PerfTrampoline_CompileCode under appropriate ifdefs

c6cf583

use PyAPI_FUNC when defining new c-api

ff5f1e4

fixup! use PyAPI_FUNC when defining new c-api

53aebc0

pablogsal enabled auto-merge (squash) October 27, 2023 03:54

pablogsal merged commit 21f068d into python:main Oct 27, 2023

bedevere-app Bot removed the awaiting merge label Oct 27, 2023

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-109587 : Allow "precompiled" perf-trampolines to largely miti…

ce85899

…gate the cost of enabling perf-trampolines (python#109666)

colesbury mentioned this pull request May 2, 2024

Remove stray __cplusplus guard in sysmodule.c #118511

Merged

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-109587 : Allow "precompiled" perf-trampolines to largely miti…

f46131c

…gate the cost of enabling perf-trampolines (python#109666)

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines by gsallam · Pull Request #109666 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines - #109666

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines#109666
pablogsal merged 32 commits into
python:mainfrom
gsallam:main

gsallam commented Sep 21, 2023 •
edited
Loading

Uh oh!

Uh oh!

Uh oh!

brandtbucher Sep 25, 2023

Uh oh!

Uh oh!

czardoz commented Oct 5, 2023

Uh oh!

czardoz commented Oct 6, 2023

Uh oh!

pablogsal left a comment

Uh oh!

pablogsal commented Oct 20, 2023

Uh oh!

pablogsal commented Oct 26, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

pablogsal commented Oct 27, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

encukou commented Jan 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

		"PyUnstable_PerfTrampoline_CompileCode",
		"PyUnstable_PerfTrampoline_SetPersistAfterFork",

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines by gsallam · Pull Request #109666 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines - #109666

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines#109666pablogsal merged 32 commits intopython:mainpython/cpython:mainfrom gsallam:maingsallam/cpython_with_perfmap_apii:mainCopy head branch name to clipboard

Conversation

gsallam commented Sep 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

brandtbucher Sep 25, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

czardoz commented Oct 5, 2023

Uh oh!

czardoz commented Oct 6, 2023

Uh oh!

pablogsal left a comment

Choose a reason for hiding this comment

Uh oh!

pablogsal commented Oct 20, 2023

Uh oh!

pablogsal commented Oct 26, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

pablogsal commented Oct 27, 2023

Uh oh!

czardoz commented Oct 27, 2023

Uh oh!

encukou commented Jan 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

gh-109587: Allow "precompiled" perf-trampolines to largely mitigate the cost of enabling perf-trampolines#109666
pablogsal merged 32 commits into
python:mainfrom
gsallam:main

gsallam commented Sep 21, 2023 •
edited
Loading