gh-143732: add specialization for `FOR_ITER` by NekoAsakura · Pull Request #148745 · python/cpython · GitHub

	main	branch	ratio
for_iter_dict_items	110.23 ms ± 1.74 ms	99.29 ms ± 1.24 ms	1.11× faster
for_iter_dict_keys	85.20 ms ± 1.03 ms	76.20 ms ± 1.02 ms	1.12× faster
for_iter_dict_values	84.91 ms ± 1.15 ms	76.27 ms ± 1.24 ms	1.11× faster
for_iter_set	96.88 ms ± 1.05 ms	87.46 ms ± 1.16 ms	1.11× faster
for_iter_reversed	79.99 ms ± 0.77 ms	72.03 ms ± 0.79 ms	1.11× faster
for_iter_enumerate	123.65 ms ± 1.94 ms	110.14 ms ± 1.82 ms	1.12× faster
for_iter_zip	131.47 ms ± 1.38 ms	122.17 ms ± 2.87 ms	1.08× faster
for_iter_list	59.21 ms ± 1.17 ms	59.48 ms ± 1.06 ms	1.00× slower
for_iter_tuple	53.78 ms ± 0.61 ms	53.77 ms ± 0.74 ms	1.00× faster
for_iter_range	72.62 ms ± 0.80 ms	72.55 ms ± 1.08 ms	1.00× faster

Issue: Broader specialization in the Specializing Adaptive Interpreter for better JIT performance #143732

LGTM, thanks for doing this!

cpython/Python/optimizer_symbols.c

Lines 725 to 726 in ad7d361

    
           case JIT_SYM_RECORDED_GEN_FUNC_TAG: 
        
               return &PyGen_Type;

We don't need _RECORD_NOS_GEN_FUNC here, because the GEN_FUNC_TAG recorded here does not contribute to the current optimization.

Every FOR_ITER specialisation's record list must be a prefix of FOR_ITER's.
_RECORD_NOS_GEN_FUNC writes a gen func or NULL to slot 0, matching what FOR_ITER_GEN reads from it.
https://github.com/python/cpython/actions/runs/24623062259/job/71997168333

We should add an assert to make sure the last uop is _RECORD_NOS_TYPE

I've a few suggestions inline.

Why volatile? It shouldn't be necessary.
Also function pointers may not be the same size as normal pointers.
Can you add assert(sizeof(iternextfunc) == sizeof(uintptr_t)); to be on the safe side.

Because mmap pages usually sit too far from tp_iternext for the offset to reach, we have to use volatile to force compiler to emit callq *%rax (target read from a register) instead of callq <rel32> (target baked in as a fixed offset). Otherwise the call jumps to the wrong address and segfaults.

Does this fail just for x86 or for AArch64 as well?

This is a bug in the JIT and we fix it, rather than add workarounds.
I think we should be adding a trampoline here, but it seems that we are not.
If you're looking at the stencils, can you see how it is being patched (what patch function is being called)?

@brandtbucher

Both. (aarch64 was tested under qemu-user)
Trampoline path doesn't appear to be designed for _JIT_* symbols:

cpython/Tools/jit/_stencils.py

Lines 434 to 446 in 7c9ad27

    
           def symbol_to_value(symbol: str) -> tuple[HoleValue, str | None]: 
        
               """ 
        
               Convert a symbol name to a HoleValue and a symbol name. 
        
               Some symbols (starting with "_JIT_") are special and are converted to their 
        
               own HoleValues. 
        
               """ 
        
               if symbol.startswith("_JIT_"): 
        
                   try: 
        
                       return HoleValue[symbol.removeprefix("_JIT_")], None 
        
                   except KeyError: 
        
                       pass 
        
               return HoleValue.ZERO, symbol

cpython/Tools/jit/_stencils.py

Lines 288 to 292 in 7c9ad27

    
           if ( 
        
               hole.kind 
        
               in {"R_AARCH64_CALL26", "R_AARCH64_JUMP26", "ARM64_RELOC_BRANCH26"} 
        
               and hole.value is HoleValue.ZERO 
        
               and hole.symbol not in self.symbols

cpython/Tools/jit/_stencils.py

Lines 305 to 309 in 7c9ad27

    
           # x86_64 Darwin trampolines for external symbols 
        
           elif ( 
        
               hole.kind == "X86_64_RELOC_BRANCH" 
        
               and hole.value is HoleValue.ZERO 
        
               and hole.symbol not in self.symbols

Even if we extended it to those, we'd need a stub per instance. I don't think it gains us anything.

We only need a trampoline if the jitted code is too far from the executable. See #148822

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be poked with soft cushions!

Documentation build overview

📚 cpython-previews | 🛠️ Build #32525858 | 📁 Comparing a0551fe against main (4b33308)

🔍 Preview build

81 files changed · + 1 added · ± 80 modified

+ Added

c-api/sentinel.html

± Modified

OK, well let's go with your approach for now. We already do the same here

I've created #149316 so we won't need the "volatile"

We need the watcher here as well.

What we do elsewhere is something like this:

bool definite = true;
PyTypeObject *type = sym_get_type(iter);
if (type == NULL) {
    type = sym_get_probable_type(iter);
    definite = false;
}
if (type != NULL) {
   // Add if not definite, otherwise NOP
   // Add watcher(s)
}

The analyzer should take the union of all the family members when considering what to record.
Can you try removing this?

I create an pr to resolve this: #149359

This looks good now, thanks.

We can improve the tooling later. The generated code is good with the workarounds.

pythongh-143732 : add specialization for FOR_ITER

dc169d8

NekoAsakura requested review from Fidget-Spinner, markshannon, savannahostrowski and tomasr8 as code owners April 19, 2026 06:19

bedevere-app Bot added the awaiting review label Apr 19, 2026

bedevere-app Bot mentioned this pull request Apr 19, 2026

Broader specialization in the Specializing Adaptive Interpreter for better JIT performance #143732

Open

NekoAsakura added 3 commits April 19, 2026 02:41

Merge branch 'main' into pythongh-143732 /for-iter-type-recording

66c11de

Make slot compatible with FOR_ITER_GEN

3a934bf

Minimise diff

5700f6a

cocolato reviewed Apr 19, 2026

View reviewed changes

cocolato added the skip news label Apr 19, 2026

Add uop assert

1f48326

markshannon requested changes May 1, 2026

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting review labels May 1, 2026

NekoAsakura added 3 commits May 1, 2026 16:15

address feedback

06ce7a4

Merge branch 'main' into pythongh-143732 /for-iter-type-recording

4003e8c

uniform nos recording across FOR_ITER

0f69071

NekoAsakura added 2 commits May 2, 2026 09:28

Merge branch 'main' into pythongh-143732 /for-iter-type-recording

724e285

Merge branch 'main' into pythongh-148871 /load-common-constant

cbd17a2

markshannon mentioned this pull request May 3, 2026

JIT does not emit trampolines for jumps to addresses encoded as operands #149316

Open

markshannon self-requested a review May 3, 2026 09:54

markshannon reviewed May 3, 2026

View reviewed changes

Comment thread Python/bytecodes.c Outdated

markshannon reviewed May 3, 2026

View reviewed changes

NekoAsakura added 2 commits May 3, 2026 10:07

address feedback

49baba9

Merge branch 'main' into pythongh-143732 /for-iter-type-recording

fc9da2c

markshannon reviewed May 4, 2026

View reviewed changes

cocolato mentioned this pull request May 4, 2026

gh-149358: Generating JIT recorder transformers and stabilizing family layouts #149359

Merged

NekoAsakura added 3 commits May 4, 2026 08:35

update family-uniform recording

a0551fe

Merge branch 'main' into pythongh-143732 /for-iter-type-recording

52e0325

update test

826b21d

markshannon self-requested a review May 4, 2026 16:26

markshannon reviewed May 4, 2026

View reviewed changes

markshannon self-requested a review May 4, 2026 16:27

markshannon approved these changes May 4, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting changes labels May 4, 2026

markshannon merged commit 9846407 into python:main May 4, 2026
86 of 87 checks passed

bedevere-app Bot removed the awaiting merge label May 4, 2026

NekoAsakura deleted the gh-143732/for-iter-type-recording branch May 4, 2026 17:34

savannahostrowski mentioned this pull request May 7, 2026

JIT hang in xml.etree.iterparse #149481

Closed

+                              PyType_Watch(TYPE_WATCHER_ID, (PyObject *)probable);
+                              _Py_BloomFilter_Add(dependencies, probable);
+                              sym_set_type(iter, probable);
+                              int32_t orig_target = (this_instr - 1)->target;

+                  op(_FOR_ITER_TIER_TWO, (iter, null_or_index -- iter, null_or_index, next)) {
+                      PyTypeObject *type = sym_get_type(iter);
+                      if (type != NULL && type != &PyGen_Type && type->tp_iternext != NULL) {
+                          ADD_OP(_ITER_NEXT_INLINE, 0, (uintptr_t)type->tp_iternext);

                       }
-                      macro(FOR_ITER) = _SPECIALIZE_FOR_ITER + _FOR_ITER;
+                      macro(FOR_ITER) = _SPECIALIZE_FOR_ITER + _RECORD_NOS_GEN_FUNC + _RECORD_NOS_TYPE + _FOR_ITER;

+                      }
+                      tier2 op(_ITER_NEXT_INLINE, (iternext_fn/4, iter, null_or_index -- iter, null_or_index, next)) {
+                          volatile iternextfunc iternext_v = (iternextfunc)iternext_fn;

	def symbol_to_value(symbol: str) -> tuple[HoleValue, str \| None]:
	"""
	Convert a symbol name to a HoleValue and a symbol name.

	Some symbols (starting with "_JIT_") are special and are converted to their
	own HoleValues.
	"""
	if symbol.startswith("_JIT_"):
	try:
	return HoleValue[symbol.removeprefix("_JIT_")], None
	except KeyError:
	pass
	return HoleValue.ZERO, symbol

	if (
	hole.kind
	in {"R_AARCH64_CALL26", "R_AARCH64_JUMP26", "ARM64_RELOC_BRANCH26"}
	and hole.value is HoleValue.ZERO
	and hole.symbol not in self.symbols

	# x86_64 Darwin trampolines for external symbols
	elif (
	hole.kind == "X86_64_RELOC_BRANCH"
	and hole.value is HoleValue.ZERO
	and hole.symbol not in self.symbols

                       macro(FOR_ITER_VIRTUAL) =
                           unused/1 +  // Skip over the counter
+                          _RECORD_NOS +  // Required for family-uniform recording (gh-148571).

gh-143732: add specialization for `FOR_ITER` by NekoAsakura · Pull Request #148745 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-143732: add specialization for FOR_ITER - #148745

gh-143732: add specialization for FOR_ITER#148745markshannon merged 15 commits intopython:mainpython/cpython:mainfrom NekoAsakura:gh-143732/for-iter-type-recordingNekoAsakura/cpython:gh-143732/for-iter-type-recordingCopy head branch name to clipboard

Conversation

NekoAsakura commented Apr 19, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cocolato left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cocolato Apr 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

markshannon May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented May 1, 2026

Uh oh!

read-the-docs-community Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

markshannon commented May 3, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gh-143732: add specialization for FOR_ITER#148745
markshannon merged 15 commits into
python:mainfrom
NekoAsakura:gh-143732/for-iter-type-recording

NekoAsakura commented Apr 19, 2026 •
edited by bedevere-app Bot
Loading

cocolato Apr 19, 2026 •
edited
Loading

markshannon May 1, 2026 •
edited
Loading

read-the-docs-community Bot commented May 2, 2026 •
edited
Loading