Add c-api to set callback function on enter and exit of PyContext · Issue #119333 · python/cpython · GitHub

Feature or enhancement

Proposal:

In python every time a context is switched, two c-api are called.

PyContext_Enter(PyObject *);
PyContext_Exit(PyObject *);

I need a way to know when those api's are called on a context. The purpose is that at Meta we have a similar system to contextvars in C++ called folly::requestcontext. When the python context switches we need to inform C++ to swap its active context. Right now we have been carry forward patches to Asyncio.Task to do this on the _step() since way back in py3.6 which is generally the wrong place to do it.

There is prior art in #91054 which provides watching creation/destruction of Code objects. The proposed solution would mirror that functionality closely.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/request-can-we-get-a-c-api-hook-into-pycontext-enter-and-pycontext-exit/51730

There was an in person meeting between @gvanrossum, @itamaro and myself where we discussed the plausibility of this feature request.

Linked PRs

gh-119333: Add C api to have contextvar enter/exit callbacks (#119335)

gh-119333: get interp from tstate for PyContext watchers(#124444)

@fried I'm not sure that context entry and exit are the appropriate semantics for this. Don't you actually want to know when the "current context" switches to a different context?

Why this matters: #99633 is proposing to add the ability to use contextvars.Context as a context manager, which involves temporarily changing the "current context" (but not necessarily entering and exiting contexts) while a generator or coroutine is executing. See #99633 (comment) for some technical details.

pythongh-119333: Change PyContext_WatchCallback to take PyObject

pythongh-119333: Back up exception before calling PyContext_WatchCall…

pythongh-119333, pythongh-99633: Refine contextvars documentation