A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

BTW, JIT CI is broken on macOS and Windows currently, so if those fail (build failure on Windows and 2 failing tests on macOS), it's not your fault.

Technically, this should probably also support CFLAGS_NODIST? I'm not sure.

It's up to you. If you want to add CFLAGS_NODIST, just use the same --with-cflags option to pass them though (the build script doesn't care which is which).

So, this breaks on universal builds, since that sets CONFIGURE_CFLAGS= -arch arm64 -arch x86_64. See the AArch64 macOS failures in CI (it's different from the expected test failures).

I think part of the problem is that we don't want to use an option for the JIT just because it's being used for the interpreter itself. We want very tight control over the code we're generating, especially since the parse step is sort of fragile.

What flags are you needing to pass, exactly? Is it only include paths?

I only need the include flag for my build to pass. But I only caught #134291 because all of my flags were passed through. It's possible there are other subtle differences. I'm totally fine with a CFLAGS_JIT, personally. I feel like people are opposed to new CFLAGS variants, but here it makes a lot of sense.

I've updated to use CFLAGS_JIT

@brandtbucher gentle nudge, so I don't need to maintain patches in perpetuity :)

Thanks! I just pushed the last change myself.

Sorry, I just pulled this down to play with it locally, and I have one question. It seems suboptimal to have to set the environment variable when actually running make. It think the way we prefer to do things like this is to "lock-in" configuration like this at configure time instead. For example, this won't currently work:

$ CFLAGS_JIT="-Icustom/include/dir" ./configure --enable-experimental-jit
$ # ...some time later...
$ make

What do you think about this patch, which will capture the variable at configure time and hardcode it in the Makefile? I think this is closer to what we do for similar things like CFLAGS_NODIST. I'm not an Autoconf or build system expert though, so I could be missing something:

diff --git a/configure.ac b/configure.ac
index 9ab4b370f1c..f6091c21b2a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2752,7 +2752,6 @@ AS_VAR_IF([ac_cv_gcc_compat], [yes], [
 ])
 
 # Check for --enable-experimental-jit:
-AC_SUBST([CFLAGS_JIT])
 AC_MSG_CHECKING([for --enable-experimental-jit])
 AC_ARG_ENABLE([experimental-jit],
               [AS_HELP_STRING([--enable-experimental-jit@<:@=no|yes|yes-off|interpreter@:>@],
@@ -2777,7 +2776,7 @@ AS_VAR_IF([jit_flags],
           [],
           [AS_VAR_APPEND([CFLAGS_NODIST], [" $jit_flags"])
            AS_VAR_SET([REGEN_JIT_COMMAND],
-                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"\$(CFL
AGS_JIT)\""])
+                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAG
S_JIT\""])
            AS_VAR_SET([JIT_STENCILS_H], ["jit_stencils.h"])
            AS_VAR_IF([Py_DEBUG],
                      [true],
diff --git a/configure b/configure
index 68213817679..6170e67e6e2 100755
--- a/configure
+++ b/configure
@@ -906,7 +906,6 @@ DSYMUTIL_PATH
 DSYMUTIL
 JIT_STENCILS_H
 REGEN_JIT_COMMAND
-CFLAGS_JIT
 UNIVERSAL_ARCH_FLAGS
 WASM_STDLIB
 WASM_ASSETS_DIR
@@ -10833,7 +10832,6 @@ esac
 fi
 
 # Check for --enable-experimental-jit:
-
 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --enable-experimental-jit" >&5
 printf %s "checking for --enable-experimental-jit... " >&6; }
 # Check whether --enable-experimental-jit was given.
@@ -10865,7 +10863,7 @@ then :
 
 else case e in #(
   e) as_fn_append CFLAGS_NODIST " $jit_flags"
-           REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"
\$(CFLAGS_JIT)\""
+           REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"
$CFLAGS_JIT\""
            JIT_STENCILS_H="jit_stencils.h"
            if test "x$Py_DEBUG" = xtrue
 then :

t seems suboptimal to have to set the environment variable when actually running make. It think the way we prefer to do things like this is to "lock-in" configuration like this at configure time instead.

Oh, that's how I thought it was working originally and definitely my intent. I guess I broke that when I changed it to CFLAGS_JIT, thanks for checking!

What do you think about this patch, which will capture the variable at configure time and hardcode it in the Makefile?

If the patch works, that's great.

(I can also actually use this patch in python-build-standalone to confirm it's working as intended, right now I'm using something hackier)

@zanieb did you get a chance to try this out? I'll land if so.

I didn't try it yet, I can do so now.

Seems happy astral-sh/python-build-standalone#659

@hugovk could we have this in 3.14 please? This is ostensively a bug in the build system and I have to maintain this patch downstream.

@zanieb Yes, that's okay.

Thanks @zanieb for the PR, and @brandtbucher for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

Reminder about backporting. @zanieb @brandtbucher

GH-135792 is a backport of this pull request to the 3.14 branch.