| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| Expand Up | @@ -102,7 +102,7 @@ struct _is { | |
| In order to be effective, this must be set to 0 during or right | ||
| after allocation. */ | ||
| int _initialized; | ||
| int finalizing; | ||
| int started_finalization; /* set early, used to block some actions. */ | ||
|
Comment thread
Copy link
Copy Markdown
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityCan you give some examples here of actions which are blocked?
Sorry, something went wrong.
All reactions
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityDepending upon implementation i'm not sure we even want this flag anymore... If we allow threading and forking to happen up until the modern _finalizing API returns False there'd be no need for it. Otherwise "some actions" needing describing is basically those two and an odd debug thing in unicodeobject.c as far the users of this go. Every place that touches it can be seen in this PR.
Sorry, something went wrong.
All reactions
|
||
|
|
||
| uintptr_t last_restart_version; | ||
| struct pythreads { | ||
| Expand All | @@ -125,7 +125,7 @@ struct _is { | |
| Get runtime from tstate: tstate->interp->runtime. */ | ||
| struct pyruntimestate *runtime; | ||
|
|
||
| /* Set by Py_EndInterpreter(). | ||
| /* Set by Py_EndInterpreter() & Py_FinalizeEx(). | ||
|
|
||
| Use _PyInterpreterState_GetFinalizing() | ||
| and _PyInterpreterState_SetFinalizing() | ||
| Expand Down | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| Expand Up | @@ -1515,22 +1515,19 @@ def _shutdown(): | |
| """ | ||
| Wait until the Python thread state of all non-daemon threads get deleted. | ||
| """ | ||
| # Obscure: other threads may be waiting to join _main_thread. That's | ||
| # dubious, but some code does it. We can't wait for it to be marked as done | ||
| # normally - that won't happen until the interpreter is nearly dead. So | ||
| # mark it done here. | ||
| if _main_thread._handle.is_done() and _is_main_interpreter(): | ||
|
Comment thread
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityThis early return logic started in ee84a60 but no longer seems relevant in 3.13 given the other refactoring in threading.py cleaning up the locking mess.
Sorry, something went wrong.
All reactions
|
||
| # _shutdown() was already called | ||
| return | ||
|
|
||
| global _SHUTTING_DOWN | ||
| _SHUTTING_DOWN = True | ||
|
|
||
| # Call registered threading atexit functions before threads are joined. | ||
| # Order is reversed, similar to atexit. | ||
| for atexit_call in reversed(_threading_atexits): | ||
| while _threading_atexits and (atexit_call := _threading_atexits.pop()): | ||
|
Comment thread
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality_shutdown can be called twice given the re-join after atexit handlers if other threads are found. removing them as they're called prevents future calls. I should turn this into a comment. test_atexit already has tests that cover this situation.
Sorry, something went wrong.
All reactions
|
||
| atexit_call() | ||
|
|
||
| # Obscure: Other threads may be waiting to join _main_thread. That's | ||
| # dubious, but some code does it. They can't wait for it to be marked done | ||
| # normally - that won't happen until the interpreter is nearly dead. So | ||
| # mark it done here so they can proceed and be joined below. | ||
| # BPO-18984's https://github.com/python/cpython/commit/c363a23eff25d8ee74d | ||
| if _is_main_interpreter(): | ||
| _main_thread._handle._set_done() | ||
|
|
||
| Expand Down | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| Restores the ability to spawn threads from :mod:`atexit` handlers. The | ||
| interpreter finalization code will join non-daemon threads spawned from that | ||
| context before exiting. No atexit handlers can be registered once atexit | ||
| handler processing has started and none will remain to run after threads | ||
| spawned during atexit processing exit. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| Expand Up | @@ -87,6 +87,7 @@ _PyAtExit_Init(PyInterpreterState *interp) | |
|
|
||
| state->callback_len = 32; | ||
| state->ncallbacks = 0; | ||
| _Py_atomic_store_int(&state->handler_calling_started, 0); | ||
| state->callbacks = PyMem_New(atexit_py_callback*, state->callback_len); | ||
| if (state->callbacks == NULL) { | ||
| return _PyStatus_NO_MEMORY(); | ||
| Expand Down Expand Up | @@ -118,10 +119,14 @@ _PyAtExit_Fini(PyInterpreterState *interp) | |
|
|
||
|
|
||
| static void | ||
| atexit_callfuncs(struct atexit_state *state) | ||
| atexit_callfuncs(struct atexit_state *state, int from_python_module) | ||
| { | ||
| assert(!PyErr_Occurred()); | ||
|
|
||
| if (!from_python_module) { | ||
|
Comment thread
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Qualitybecause Lib/_test_atexit.py calls atexit._run_callbacks() which shouldn't set the flag.
Sorry, something went wrong.
All reactions
|
||
| _Py_atomic_store_int(&state->handler_calling_started, 1); | ||
| } | ||
|
|
||
| if (state->ncallbacks == 0) { | ||
| return; | ||
| } | ||
| Expand Down Expand Up | @@ -155,7 +160,7 @@ void | |
| _PyAtExit_Call(PyInterpreterState *interp) | ||
| { | ||
| struct atexit_state *state = &interp->atexit; | ||
| atexit_callfuncs(state); | ||
| atexit_callfuncs(state, 0); | ||
| } | ||
|
|
||
|
|
||
| Expand Down Expand Up | @@ -191,6 +196,29 @@ atexit_register(PyObject *module, PyObject *args, PyObject *kwargs) | |
| } | ||
|
|
||
| struct atexit_state *state = get_atexit_state(); | ||
| if (_Py_atomic_load_int(&state->handler_calling_started)) { | ||
|
Comment thread
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Qualityarguably all of the logic around blocking new atexit.registers from happening could be left out of this PR. it felt relevant though. we never really defined or documented what happened before (registrations made from a handler were never called). This makes it explicit and gives some indication as to when a registration is being ignored.
Sorry, something went wrong.
All reactions
|
||
| /* We could instead raise an error in this situation, but it is action | ||
| * at a distance. Not the callers fault. The culprit may be via a call | ||
| * chain invoking code transitively via an atexit handler or from | ||
| * within a thread spawned by an atexit handler. See GH-113964. | ||
| */ | ||
| if (PyErr_WarnEx( | ||
| PyExc_RuntimeWarning, | ||
| "atexit.register() ignored; atexit handler calls" | ||
| " have already started.", | ||
| 1)) { | ||
| return NULL; | ||
| } | ||
| return Py_NewRef(func); | ||
| } | ||
|
|
||
| // Extremely unlikely, just to avoid overflow. | ||
| if (state->ncallbacks > INT_MAX/2048) { | ||
| PyErr_SetString(PyExc_RuntimeError, | ||
| "Too many atexit handlers registered."); | ||
| return NULL; | ||
| } | ||
|
|
||
| if (state->ncallbacks >= state->callback_len) { | ||
| atexit_py_callback **r; | ||
| state->callback_len += 16; | ||
| Expand Down Expand Up | @@ -231,7 +259,7 @@ static PyObject * | |
| atexit_run_exitfuncs(PyObject *module, PyObject *unused) | ||
| { | ||
| struct atexit_state *state = get_atexit_state(); | ||
| atexit_callfuncs(state); | ||
| atexit_callfuncs(state, 1); | ||
| Py_RETURN_NONE; | ||
| } | ||
|
|
||
| Expand Down | ||
|
Comment thread
Copy link
Copy Markdown
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityThere's a lot of duplication between Py_FinalizeEx and PyEnd_Interpreter but this PR is not the place to reconcile any of that.
Sorry, something went wrong.
All reactions
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| Expand Up | @@ -1877,27 +1877,27 @@ Py_FinalizeEx(void) | |
| // XXX assert(_Py_IsMainInterpreter(tstate->interp)); | ||
| // XXX assert(_Py_IsMainThread()); | ||
|
|
||
| // Block some operations. | ||
| tstate->interp->finalizing = 1; | ||
| // Block some operations including fork(). | ||
| tstate->interp->started_finalization = 1; | ||
|
|
||
| // Wrap up existing "threading"-module-created, non-daemon threads. | ||
| wait_for_thread_shutdown(tstate); | ||
|
|
||
| // Make any remaining pending calls. | ||
| _Py_FinishPendingCalls(tstate); | ||
|
|
||
| /* The interpreter is still entirely intact at this point, and the | ||
| * exit funcs may be relying on that. In particular, if some thread | ||
| * or exit func is still waiting to do an import, the import machinery | ||
| * expects Py_IsInitialized() to return true. So don't say the | ||
| * runtime is uninitialized until after the exit funcs have run. | ||
| * Note that Threading.py uses an exit func to do a join on all the | ||
| * threads created thru it, so this also protects pending imports in | ||
| * the threads created via Threading. | ||
| * atexit handlers are relying on that. We cannot say the runtime is | ||
| * uninitialized until after the atexit handlers have run and and any | ||
| * non-daemon threads they spawned have been joined. | ||
| */ | ||
|
|
||
| _PyAtExit_Call(tstate->interp); | ||
|
|
||
| if (tstate != tstate->interp->threads.head || tstate->next != NULL) { | ||
|
Comment thread
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityI think if you are accessing tstate->interp->threads.head or tstate->next you need to be within a HEAD_LOCK(&_PyRuntime);.
Sorry, something went wrong.
gpshead reacted with thumbs up emoji
All reactions
|
||
| // An atexit or signal handler must've spawned a thread; one last reap. | ||
| wait_for_thread_shutdown(tstate); | ||
| _Py_FinishPendingCalls(tstate); | ||
| } | ||
|
|
||
| /* Copy the core config, PyInterpreterState_Delete() free | ||
| the core config memory */ | ||
| #ifdef Py_REF_DEBUG | ||
| Expand Down Expand Up | @@ -2259,18 +2259,23 @@ Py_EndInterpreter(PyThreadState *tstate) | |
| if (tstate->current_frame != NULL) { | ||
| Py_FatalError("thread still has a frame"); | ||
| } | ||
| interp->finalizing = 1; | ||
| // Block some operations including fork(). | ||
| interp->started_finalization = 1; | ||
|
|
||
| // Wrap up existing "threading"-module-created, non-daemon threads. | ||
| wait_for_thread_shutdown(tstate); | ||
|
|
||
| // Make any remaining pending calls. | ||
| _Py_FinishPendingCalls(tstate); | ||
|
|
||
| _PyAtExit_Call(tstate->interp); | ||
|
|
||
| if (tstate != interp->threads.head || tstate->next != NULL) { | ||
| Py_FatalError("not the last thread"); | ||
| // An atexit or signal handler must've spawned a thread; one last reap. | ||
| wait_for_thread_shutdown(tstate); | ||
| _Py_FinishPendingCalls(tstate); | ||
| if (tstate != interp->threads.head || tstate->next != NULL) { | ||
| Py_FatalError("not the last thread"); | ||
| } | ||
| } | ||
|
|
||
| /* Remaining daemon threads will automatically exit | ||
| Expand Down | ||
| Back | FazBrowse Home | New Git URL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityThis seemed to be a more accurate name given what it gets used for and the below more comprehensive _finalizing with an API around it.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.