FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-139540: Fix executor deallocation crash when JIT compilation fails by ashm-dev · Pull Request #139952 · python/cpython · GitHub

/ cpython Public

gh-139540: Fix executor deallocation crash when JIT compilation fails #139952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ashm-dev wants to merge 3 commits into from
Conversation 19 Commits 3 Checks 104 Files changed
Closed

gh-139540: Fix executor deallocation crash when JIT compilation fails #139952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3a2d34b
Track executor object with GC
ashm-dev Oct 11, 2025
963bcf0
Create 2025-10-11-16-14-36.gh-issue-139549.9BGTLP.rst
ashm-dev Oct 11, 2025
db344b2
Fix memory leak in JIT executor on compilation failure
ashm-dev Oct 11, 2025
File filter

Filter by extension

Filter by extension .c  (4) .h  (2) .rst  (1) All 3 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
9 changes: 9 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2025-10-11-16-14-36.gh-issue-139549.9BGTLP.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Fix memory leak in JIT executor on compilation failure

The executor object passed to the TIER1_TO_TIER2 macro was not
decremented if the Tier 2 code returned with an exception set but
without a fatal error (i.e., not returning NULL).

This change moves the Py_DECREF call inside the TIER1_TO_TIER2
macro to ensure the executor's reference count is always decremented
after its execution, regardless of the outcome.
1 change: 1 addition & 0 deletions Python/bytecodes.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -2972,6 +2972,7 @@ dummy_func(
assert(tstate->current_executor == NULL);
assert(executor != tstate->interp->cold_executor);
tstate->jit_exit = NULL;
tstate->current_executor = (PyObject *)executor;

picnixz Oct 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Is it just to hold a reference to the executor?
TIER1_TO_TIER2(executor);
}
}
Expand Down
5 changes: 5 additions & 0 deletions Python/ceval.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1146,6 +1146,11 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, _PyInterpreterFrame *frame, int
assert(frame->owner == FRAME_OWNED_BY_INTERPRETER);
/* Restore previous frame and exit */
tstate->current_frame = frame->previous;
#ifdef _Py_TIER2
if (tstate->current_executor != NULL) {
tstate->current_executor = NULL;
}
#endif
return NULL;
}
#ifdef _Py_TIER2
Expand Down
5 changes: 2 additions & 3 deletions Python/ceval_macros.h
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -351,12 +351,12 @@ _PyFrame_SetStackPointer(frame, stack_pointer)

/* Tier-switching macros. */

#define TIER1_TO_TIER2(EXECUTOR) \
#define TIER1_TO_TIER2(EXECUTOR) \

picnixz Oct 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why was this changed? please revert.
do { \
OPT_STAT_INC(traces_executed); \

picnixz Oct 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why was this changed? please revert.
next_instr = _Py_jit_entry((EXECUTOR), frame, stack_pointer, tstate); \
frame = tstate->current_frame; \
stack_pointer = _PyFrame_GetStackPointer(frame); \
Py_DECREF(EXECUTOR); \

picnixz Oct 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why should we decref it here?
if (next_instr == NULL) { \
next_instr = frame->instr_ptr; \
JUMP_TO_LABEL(error); \
Expand Down Expand Up @@ -413,4 +413,3 @@ check_periodics(PyThreadState *tstate) {
}
return 0;
}

6 changes: 6 additions & 0 deletions Python/generated_cases.c.h
Show comments
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Python/optimizer.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1202,6 +1202,7 @@ make_executor_from_uops(_PyUOpInstruction *buffer, int length, const _PyBloomFil
if (executor == NULL) {
return NULL;
}
_PyObject_GC_TRACK(executor);

/* Initialize exits */
_PyExecutorObject *cold = _PyExecutor_GetColdExecutor();
Expand Down Expand Up @@ -1257,7 +1258,6 @@ make_executor_from_uops(_PyUOpInstruction *buffer, int length, const _PyBloomFil
return NULL;
}
#endif
_PyObject_GC_TRACK(executor);
return executor;
}

Expand Down Expand Up @@ -1507,6 +1507,7 @@ _PyExecutor_GetColdExecutor(void)
if (cold == NULL) {
Py_FatalError("Cannot allocate core JIT code");
}
_PyObject_GC_TRACK(cold);

picnixz Oct 11, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is incorrect. The executor is immortal for now.

No this was correct, my bad. I didn't read the function correctly. The problem is that we're setting it immortal afterwards and I'm not sure it will be happy.

ashm-dev Oct 11, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Good point, thanks for the correction.

The intent was to track it for consistency immediately after allocation. My assumption is the GC correctly handles a tracked object that later becomes immortal.

Is that a safe assumption, or should we explicitly _PyObject_GC_UNTRACK it just before immortalization?

picnixz Oct 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

That I don't know. The problem here is whether we want an immortal cold executor or not. That's why I wanted to wait for Mark's output here, and didn't write a PR already.
((_PyUOpInstruction *)cold->trace)->opcode = _COLD_EXIT;
#ifdef _Py_JIT
cold->jit_code = NULL;
Expand Down
3 changes: 3 additions & 0 deletions Python/pystate.c
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1717,6 +1717,9 @@ PyThreadState_Clear(PyThreadState *tstate)
Py_CLEAR(tstate->async_gen_finalizer);

Py_CLEAR(tstate->context);
#ifdef _Py_TIER2
Py_CLEAR(tstate->current_executor);
#endif

#ifdef Py_GIL_DISABLED
// Each thread should clear own freelists in free-threading builds.
Expand Down
Loading
Back | FazBrowse Home | New Git URL