gh-132732: Automatically constant evaluate pure operations by Fidget-Spinner · Pull Request #132733 · python/cpython · GitHub

Issue: Constant evaluate/propagate pure ops automatically #132732

All commit authors signed the Contributor License Agreement.

This is really neat!

Other than two opcodes I found that shouldn't be marked pure, I just have one thought:

Rather than rewriting the bodies like this to use the symbols-manipulating functions (which seems error-prone), would we be able to just use stackrefs to do this?

For example, _BINARY_OP_ADD_INT is defined like this:

PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
// ...
res = PyStackRef_FromPyObjectSteal(res_o);

Rather than rewriting uses of these functions, could it be easier to just do something like this, since we're guranteed not to escape?

if (sym_is_const(ctx, stack_pointer[-2]) && sym_is_const(ctx, stack_pointer[-1])) {
    // Generated code to turn constant symbols into stackrefs:
    _PyStackRef left = PyStackRef_FromPyObjectBorrow(sym_get_const(ctx, stack_pointer[-2]));
    _PyStackRef right = PyStackRef_FromPyObjectBorrow(sym_get_const(ctx, stack_pointer[-1]));
    _PyStackRef res;
    // Now the actual body, same as it appears in executor_cases.c.h:
    PyObject *left_o = PyStackRef_AsPyObjectBorrow(left);
    PyObject *right_o = PyStackRef_AsPyObjectBorrow(right);
    // ...
    res = PyStackRef_FromPyObjectSteal(res_o);
    // Generated code to turn stackrefs into constant symbols:
    stack_pointer[-1] = sym_new_const(ctx, PyStackRef_AsPyObjectSteal(res));
}

I'm not too familiar with the design of the cases generator though, so maybe this is way harder or something. Either way, I'm excited to see this get in!

Rather than rewriting uses of these functions, could it be easier to just do something like this, since we're guranteed not to escape?

Seems feasible. I could try to rewrite all occurences of the variable with a stackref-producing const one. Let me try that.

I've verified no refleak on test_capi.test_opt locally apart from #132731 which is pre-existing.

There's a lot going on in this PR, probably too much for one PR.

Could we start with a PR to fix up the pure annotations so that they are on the correct instructions and maybe add the pure_guard annotation that Brandt suggested?

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

Hm, I think I’d prefer not to. Sounds like it could hurt performance, especially for the JIT (where things can’t inline).

I think a good progression would be:

Implement the pure attribute, and the optimizer changes. Remove the pure attributes where they don’t belong (so nothing breaks) and leave the existing ones as proof that the implementation works. (This PR)
Audit the existing non-pure bytecodes and add pure where it makes sense. (Follow-up PR)
Implement the pure_guard attribute, and annotate any bytecodes that can use it. (Follow-up PR)

Could we have the default code generator generate a function for the body of the pure instruction and then call that from the three interpreters?

Hm, I think I’d prefer not to. Sounds like it could hurt performance, especially for the JIT (where things can’t inline).

I thought about this and I think we can inline if we autogenerate a header file and include that directly. But then we're at the mercy of the compiler in both the normal interpreter and the JIT deciding to inline or not to inline the body again. Which I truly do not want.

@brandtbucher @markshannon what can I do to get this PR moving?

@tomasr8 if youd like to review, here's a summary of the PR:

If a bytecode operation is pure (no side effects) we can mark it as pure in bytecodes.c.
In the optimizer, we automatically generate the body that does evaluation of the symbolic constants by copy pasting the bytecodes.c definition into the optimizer's C code. Of course we check that the inputs are constants first.
All changes to the cases generator is for the second point.

Thanks for the ping! I actually wanted to try/review this PR, I was just very busy this week with work :/ I'll have a look this weekend :)

Only had time to skim the PR, I'll do a more thorough review this weekend :)

Co-Authored-By: Tomas R. <tomas.roun8@gmail.com>

In case it wasn't clear. The helper functions would be:
PyObject *compare_op_0(PyObject *a, PyObject *b) { return PyObject_RichCompare(a, b, 0) }
and so on.

In case it wasn't clear. The helper functions would be: PyObject *compare_op_0(PyObject *a, PyObject *b) { return PyObject_RichCompare(a, b, 0) } and so on.

Is this the same as this?

If you're worried about the bulk of the generated code, I can open an issue for someone to make a follow up PR to generate function templates from the bytecodes.c file, similar to what we already do for the tail calling interpreter. This would allow us to just call the function. That should go into a follow-up commit though, because it requires changes that affect more than just the optimizer (it will also touch executor_cases and such).

Once skip_to is removed from generators_common, this becomes nicely self contained.

With it properly encapsulated, it should be easy to modify in the future should we decide to use a slightly different approach.

I've a few more suggestions for tidying things up, but nothing fundamental.

This isn't "pure". It can do anything

This is only used in the optimizer emitter (and I don't think it is needed there anyway)

See earlier comment.

Suggested change

      
                    emitter = OptimizerConstantEmitter(self.out, {}, self.original_uop, copy.deepcopy(self.stack))
          
                    emitter = OptimizerConstantEmitter(self.out, {}, self.original_uop, self.stack.copy())

Stacks already support copying efficiently and correctly.

If you're just going to throw away the arguments, why have them?
Either check the names match or, better still, don't require the inputs at all.

You already have the input names from the uop, [item.name for item in uop.stack.inputs]

No this is the input names from the REPLACE_OPCODE_IF_EVALUATES_PURE macro. However, now that you mentioned it, I should get the names from the replacement function, not from this. So thanks for pointing it out.

Suggested change

      
                        emitter = OptimizerEmitter(out, {}, uop, copy.deepcopy(stack))
          
                        emitter = OptimizerEmitter(out, {}, uop, stack.copy())

Suggested change

      
                if var.is_array():
          
                assert not var.is_array(), "Unsafe to convert a symbol to an array-like StackRef."

Check this in the caller if that makes sense, and raise an analysis error there so we aren't relying on assert to handle input errors.

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Thanks Mark for the review. The code is a lot cleaner now and self-contained.

I have made the requested changes; please review again

Thanks for making the requested changes!

@markshannon, @tomasr8: please review the changes made to this pull request.

That's much better. Thanks.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x (tier-3) has failed when building commit 695ab61.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/4083) and take a look at the build logs.
Check if the failure is related to this commit (695ab61) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/4083

Failed tests:

test_socketserver

Failed subtests:

test_ThreadingUnixDatagramServer - test.test_socketserver.SocketServerTest.test_ThreadingUnixDatagramServer
test_UnixDatagramServer - test.test_socketserver.SocketServerTest.test_UnixDatagramServer

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 222, in test_UnixDatagramServer
    self.run_server(socketserver.UnixDatagramServer,
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    socketserver.DatagramRequestHandler,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    self.dgram_examine)
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/support/threading_helper.py", line 66, in decorator
    return func(*args)
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 133, in run_server
    testfunc(svrcls.address_family, addr)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 160, in dgram_examine
    buf = data = receive(s, 100)
                 ~~~~~~~^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 43, in receive
    raise RuntimeError("timed out on %r" % (sock,))
RuntimeError: timed out on <socket.socket fd=23, family=1, type=2, proto=0, laddr=./test_python_x18vehwp.sock>


Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/threading.py", line 1074, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/threading.py", line 1016, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 240, in serve_forever
    self._handle_request_noblock()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 320, in _handle_request_noblock
    self.handle_error(request, client_address)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 318, in _handle_request_noblock
    self.process_request(request, client_address)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 349, in process_request
    self.finish_request(request, client_address)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 362, in finish_request
    self.RequestHandlerClass(request, client_address, self)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 768, in __init__
    self.finish()
    ~~~~~~~~~~~^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 863, in finish
    self.socket.sendto(self.wfile.getvalue(), self.client_address)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 54] Connection reset by peer
ERROR


Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 228, in test_ThreadingUnixDatagramServer
    self.run_server(socketserver.ThreadingUnixDatagramServer,
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    socketserver.DatagramRequestHandler,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                    self.dgram_examine)
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/support/threading_helper.py", line 66, in decorator
    return func(*args)
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 133, in run_server
    testfunc(svrcls.address_family, addr)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 160, in dgram_examine
    buf = data = receive(s, 100)
                 ~~~~~~~^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/test/test_socketserver.py", line 43, in receive
    raise RuntimeError("timed out on %r" % (sock,))
RuntimeError: timed out on <socket.socket fd=23, family=1, type=2, proto=0, laddr=./test_python_a8m4t39h.sock>


Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/threading.py", line 1074, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/threading.py", line 1016, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 699, in process_request_thread
    self.handle_error(request, client_address)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 697, in process_request_thread
    self.finish_request(request, client_address)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 362, in finish_request
    self.RequestHandlerClass(request, client_address, self)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 768, in __init__
    self.finish()
    ~~~~~~~~~~~^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/3299090E-A190-41F9-9EFB-E7EB12DA3473/data/Containers/Bundle/Application/EDDB3F71-0E9C-40F0-97D8-EB8A643BB963/iOSTestbed.app/python/lib/python3.15/socketserver.py", line 863, in finish
    self.socket.sendto(self.wfile.getvalue(), self.client_address)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 54] Connection reset by peer
Warning -- threading_cleanup() failed to clean up threads in 1.0 seconds
Warning --   before: thread count=0, dangling=1
Warning --   after: thread count=1, dangling=4
Warning -- Dangling thread: <_MainThread(MainThread, started 4302897664)>
Warning -- Dangling thread: <Thread(Thread-2933 (process_request_thread), stopped 6218264576)>
Warning -- Dangling thread: <Thread(Thread-2932 (process_request_thread), stopped 6201438208)>
Warning -- Dangling thread: <Thread(<class 'socketserver.ThreadingUnixDatagramServer'> serving, started daemon 6184611840)>
ERROR

@Fidget-Spinner, tuple isn't safe for comparisons either. The contained items may escape when compared.

We could check the items recursively for safety using sym_tuple_getitem. Or just remove this.

I'm leaning slightly towards the recursive check, but either could make sense.

We should also add None here, as well.

Ok let's remove it.

…honGH-132733) This adds a "macro" to the optimizer DSL called "REPLACE_OPCODE_IF_EVALUATES_PURE", which allows automatically constant evaluating a bytecode body if certain inputs have no side effects upon evaluations (such as ints, strings, and floats). Co-authored-by: Tomas R. <tomas.roun8@gmail.com>

Automatically constant evaluate pure operations

1ffbb6b

Fidget-Spinner requested a review from markshannon as a code owner April 19, 2025 16:21

bedevere-app Bot added the awaiting core review label Apr 19, 2025

Fidget-Spinner requested review from brandtbucher and removed request for markshannon April 19, 2025 16:21

bedevere-app Bot mentioned this pull request Apr 19, 2025

Constant evaluate/propagate pure ops automatically #132732

Closed

blurb-it Bot and others added 3 commits April 19, 2025 16:22

📜🤖 Added by blurb_it.

691084d

Fix tests

b89e4dc

Merge branch 'pure' of github.com:Fidget-Spinner/cpython into pure

0959918

StanFromIreland reviewed Apr 19, 2025

View reviewed changes

Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-04-19-16-22-47.gh-issue-132732.jgqhlF.rst

brandtbucher mentioned this pull request Apr 24, 2025

gh-131798: Use sym_new_type instead of sym_new_not_null for _BUILD_STRING, _BUILD_SET #132564

Merged

brandtbucher reviewed Apr 24, 2025

View reviewed changes

Comment thread Python/bytecodes.c Outdated

Comment thread Python/bytecodes.c Outdated

Comment thread Python/bytecodes.c Outdated

Fidget-Spinner added 2 commits April 25, 2025 07:16

Merge remote-tracking branch 'upstream/main' into pure

2541683

Apply review suggestions

d5b2208

reduce diff

71ced86

Fidget-Spinner mentioned this pull request Apr 26, 2025

gh-131798: JIT: Propagate the result in _BINARY_OP_SUBSCR_TUPLE_INT #133003

Merged

Fidget-Spinner added 2 commits May 7, 2025 06:48

Merge remote-tracking branch 'upstream/main' into pure

a10d5a1

Update pycore_opcode_metadata.h

d22f165

tomasr8 reviewed May 9, 2025

View reviewed changes

Comment thread Python/optimizer_bytecodes.c Outdated

Comment thread Python/optimizer_analysis.c Outdated

Comment thread Tools/cases_generator/optimizer_generator.py Outdated

Comment thread Tools/cases_generator/optimizer_generator.py Outdated

Apply changes from code review

8ae38c7

Co-Authored-By: Tomas R. <tomas.roun8@gmail.com>

python deleted a comment from Rk13-gold Jun 20, 2025

markshannon requested changes Jun 27, 2025

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting change review labels Jun 27, 2025

Fidget-Spinner added 3 commits June 27, 2025 17:48

Address review

0e9ce84

reduce diff

f2fc3fc

fix mypy

2408fb0

bedevere-app Bot added awaiting change review and removed awaiting changes labels Jun 27, 2025

bedevere-app Bot requested review from markshannon and tomasr8 June 27, 2025 09:51

markshannon approved these changes Jun 27, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting change review labels Jun 27, 2025

fix wrong error message

e40cb06

Fidget-Spinner merged commit 695ab61 into python:main Jun 27, 2025

bedevere-app Bot removed the awaiting merge label Jun 27, 2025

Fidget-Spinner deleted the pure branch June 27, 2025 11:37

brandtbucher reviewed Jun 27, 2025

View reviewed changes

brandtbucher mentioned this pull request Jul 1, 2025

The JIT's "pure" _BINARY_OP evaluation produces UB (and incorrect results) #136183

Closed

                   write_header,
                   Emitter,
                   TokenIterator,
+                  skip_to,

+                      storage: Storage,
+                      inst: Instruction | None,
+                  ) -> bool:
+                      skip_to(tkn_iter, "SEMI")

                                   out.emit(f"{type}{cache.name} = ({cast})this_instr->operand0;\n")
                       if override:
-                          emitter = OptimizerEmitter(out, {})
+                          emitter = OptimizerEmitter(out, {}, uop, copy.deepcopy(stack))

+                  return (typ == &PyLong_Type) ||
+                         (typ == &PyUnicode_Type) ||
+                         (typ == &PyFloat_Type) ||
+                         (typ == &PyTuple_Type) ||

                       }
-                      pure op(_BINARY_OP_EXTEND, (descr/4, left, right -- res)) {
+                     pure op(_BINARY_OP_EXTEND, (descr/4, left, right -- res)) {

                       out.emit(tkn)
                   raise analysis_error(f"Expecting {end}. Reached end of file", tkn)
+              def skip_to(tkn_iter: TokenIterator, end: str) -> Token:

	emitter = OptimizerConstantEmitter(self.out, {}, self.original_uop, copy.deepcopy(self.stack))
	emitter = OptimizerConstantEmitter(self.out, {}, self.original_uop, self.stack.copy())

               """
               import argparse
+              import copy

		return not unconditional


		def replace_opcode_if_evaluates_pure_identifiers(uop: Uop) -> list[str]:

	emitter = OptimizerEmitter(out, {}, uop, copy.deepcopy(stack))
	emitter = OptimizerEmitter(out, {}, uop, stack.copy())

                   return "JitOptRef "
+              def stackref_type_name(var: StackItem) -> str:
+                  if var.is_array():

	if var.is_array():
	assert not var.is_array(), "Unsafe to convert a symbol to an array-like StackRef."

gh-132732: Automatically constant evaluate pure operations by Fidget-Spinner · Pull Request #132733 · python/cpython · GitHub

Repository navigation

gh-132732: Automatically constant evaluate pure operations - #132733

gh-132732: Automatically constant evaluate pure operations#132733Fidget-Spinner merged 58 commits intopython:mainpython/cpython:mainfrom Fidget-Spinner:pureFidget-Spinner/cpython:pureCopy head branch name to clipboard

Conversation

Fidget-Spinner commented Apr 19, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

python-cla-bot Bot commented Apr 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

brandtbucher left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Fidget-Spinner commented Apr 24, 2025

Uh oh!

Fidget-Spinner commented Apr 25, 2025

Uh oh!

markshannon commented Apr 30, 2025

Uh oh!

markshannon commented Apr 30, 2025

Uh oh!

brandtbucher commented Apr 30, 2025

Uh oh!

brandtbucher commented Apr 30, 2025

Uh oh!

Fidget-Spinner commented Apr 30, 2025

Uh oh!

Fidget-Spinner commented May 8, 2025

Uh oh!

tomasr8 commented May 8, 2025

Uh oh!

tomasr8 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markshannon commented Jun 20, 2025

Uh oh!

Fidget-Spinner commented Jun 20, 2025

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Jun 27, 2025

Uh oh!

Fidget-Spinner commented Jun 27, 2025

Uh oh!

bedevere-app Bot commented Jun 27, 2025

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Jun 27, 2025

gh-132732: Automatically constant evaluate pure operations#132733
Fidget-Spinner merged 58 commits into
python:mainfrom
Fidget-Spinner:pure

Fidget-Spinner commented Apr 19, 2025 •
edited by bedevere-app Bot
Loading

python-cla-bot Bot commented Apr 19, 2025 •
edited
Loading