| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
There was a problem hiding this comment.
Even without deferred refcounts, this is a simplification in my eyes (and also brings Tier 1 and Tier 2 representations closer together), so we should do this regardless.
Sorry, something went wrong.
## Description This PR adds support for Python 3.14 in the profiler by updating it to handle CPython internal changes. ### Key CPython changes addressed **`_PyInterpreterFrame` Structure Changes** 1. Moved from `Include/internal/pycore_frame.h` to `Include/internal/pycore_interpframe_structs.h` 2. `PyObject *f_executable` and `PyObject *f_funcobj` changed to `_PyStackRef` type. Profilers like us now need to clear the LSB of these fields to get the `PyObject*`. See python/cpython#123923 for details 3. `int stacktop` field removed, replaced with `_PyStackRef *stackpointer` pointer. See python/cpython#121923 (GH-120024) for details 4. `PyObject *localsplus[1]` changed to `_PyStackRef localsplus[1]`. See python/cpython#118450 (gh-117139) for details **`FutureObj`/`TaskObj` Changes** 1. Added fields: `awaited_by`, `is_task`, `awaited_by_is_set` in `FutureObj_HEAD` macro 2. Added `struct llist_node_task_node` field for linked-list storage **Asyncio Task Storage Changes** Prior to Python 3.14, - All tasks are stored in `_scheduled_tasks` WeakSet ([exported](https://github.com/python/cpython/blob/e96367da1fdc1e1cf17ca523e93a127b1961b443/Modules/_asynciomodule.c#L3738) from C extension) - Eager tasks are stored in `_eager_tasks` set ([exported](https://github.com/python/cpython/blob/e96367da1fdc1e1cf17ca523e93a127b1961b443/Modules/_asynciomodule.c#L3742) from C extension) From Python 3.14, - Native `asyncio.Tasks` are stored in a linked-list (`struct llist_node`) per thread and per interpreter - [Per-thread](https://github.com/python/cpython/blob/0114178911f8713bfcb935ff5542fe61b4a5d551/Include/internal/pycore_tstate.h#L46): `tstate->asyncio_tasks_head` (in `_PyThreadStateImpl`) - [Per-interpreter](https://github.com/python/cpython/blob/0114178911f8713bfcb935ff5542fe61b4a5d551/Include/internal/pycore_interp_structs.h#L897): `interp->asyncio_tasks_head` (for lingering tasks) - Each `TaskObj` has a `task_node` field with `next` and `prev` pointers - Third-party tasks: Still stored in Python-level `_scheduled_tasks` WeakSet (now Python-only, not exported from C extension) - Eager tasks: Still stored in Python-level `_eager_tasks` set ### Implementation Summary - **Frame reading** (`frame.h`, `frame.cc`): Updated header includes to use `pycore_interpframe_structs.h` for Python 3.14+. Implemented tagged pointer handling: clear LSB of `f_executable` to recover `PyObject*` (per gh-123923). Replaced `stacktop` field access with `stackpointer` pointer arithmetic for stack depth calculation. Updated `PyGen_yf()` to use `_PyStackRef` and `stackpointer[-1]` instead of `localsplus[stacktop-1]`. Added handling for `FRAME_OWNED_BY_INTERPRETER` frame type (introduced in 3.14). - **Task structures** (`cpython/tasks.h`): Added Python 3.14+ `FutureObj_HEAD` macro with new fields: `awaited_by`, `is_task`, `awaited_by_is_set`. Added `struct llist_node task_node` field to `TaskObj` for linked-list storage. Updated `PyGen_yf()` implementation to handle `_PyStackRef` and `stackpointer` instead of `stacktop`. - **Asyncio discovery** (`tasks.h`, `threads.h`): Implemented `get_tasks_from_linked_list()` to safely iterate over circular linked-lists with iteration limits (`MAX_ITERATIONS = 2 << 15`). Added `get_tasks_from_thread_linked_list()` to read tasks from `_PyThreadStateImpl.asyncio_tasks_head` (per-thread active tasks). Added `get_tasks_from_interpreter_linked_list()` to read lingering tasks from `PyInterpreterState.asyncio_tasks_head` (per-interpreter). Updated `get_all_tasks()` to handle both linked-list (native `asyncio.Task` instances) and WeakSet (third-party tasks). - **Python integration** (`_asyncio.py`): Added compatibility handling for `BaseDefaultEventLoopPolicy` → `_BaseDefaultEventLoopPolicy` rename in 3.14. Updated `_scheduled_tasks` access to handle Python-only WeakSet (no longer exported from C extension in 3.14+). ## Testing All existing tests pass except for tests/profiling/collector/test_memalloc.py which needed some edits. ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes --------- Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com>
| Back | FazBrowse Home | New Git URL |
To support deferred reference counting, the stack pointer will need to be spilled around many calls.
The current implementation of the stack pointer as an offset from the 0th local will be too slow for that as each spill and reload will need to perform a calculation of the stack pointer from the stacktop or vice-versa.
This PR changes the stacktop offset to stackpointer pointer, so that no calculation is needed for the spill.