…protection to fiber stacks (#64772)

# Description

On Python 3.14 + Linux, every async-actor task permanently leaks ~518
KiB of live malloc (the per-task `asyncio.Task`,
`concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack
`Packer`s with 256 KiB internal buffers).
Closes #63290

### Root cause

**1. CPython 3.14 changed how it avoids stack overflow when freeing
objects.** Freeing one object can recursively free many others (a dict
frees its values, which free their contents, …), and each level is a
nested C call. To keep that from overflowing the C stack, CPython has
long had a safety mechanism (the "trashcan"): when it decides it's too
deep, it doesn't free the object right away. Instead it parks the object
on a per-thread *delete-later* list and drains the list once there's
stack headroom again. Up to 3.13, "too deep" was a simple recursion
counter. In 3.14 it's decided by comparing the actual machine **stack
pointer** against the stack bounds CPython recorded for the thread when
it attached (from pthreads, on Linux).

**2. Ray async actors don't run task code on the thread's normal
stack.** Each task executes on a small 256 KiB boost fiber stack
allocated elsewhere in memory. The problem is that CPython still thinks
the thread runs on its original pthread stack.

So while a task runs on a fiber, every "am I near the stack limit?"
check compares the fiber's stack pointer against the *pthread* stack's
bounds. On Linux, fiber stacks happen to be allocated at lower addresses
than the pthread stack, so CPython concludes the stack is hopelessly
overflowed and parks **every** object freed during the task (including
return-value serialization and end-of-task cleanup) on the delete-later
list.

That list is only ever drained by a later free on the same thread state
at a healthy stack margin, which never happens here as the Ray thread
only runs on fibers and Ray creates a fresh Python thread state per task
and destroys it at task end. This means that CPython destroys a thread
state **without draining its delete-later list** and the parked objects
are orphaned permanently. That's the leak.

Why the confusing symptoms:

- `boost::make_fcontext` in the issue's flamegraphs just marks *where*
the leaked allocations were made (on a fiber stack); the fiber stacks
themselves are freed correctly.
- macOS is unaffected only by luck: fiber stacks there land at *higher*
addresses than the pthread stack, so the check passes.
- 3.13 and earlier are unaffected because their trashcan uses the
counter, not the stack pointer.

### Fix

CPython 3.14.2 added an official API for exactly this situation:
`PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets
an embedder tell CPython "this thread is currently executing on *this*
stack." We call it with the fiber's stack bounds:

- at async-actor task entry in `task_execution_handler`, and
- whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers
share the thread state, so each must re-register its own stack).

With the bounds correct, the near-limit check returns to normal
behavior: objects are freed immediately, and the rare genuinely-deep
free is parked and then properly drained.

Implementation notes: the symbol is looked up via `dlsym`, so `_raylet`
still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a
more severe, since-fixed stack-check bug anyway, python/cpython#141944).
No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are
derived from the current stack pointer minus a conservative allowance
for stack already used, so the protection errs toward triggering
slightly early rather than missing an overflow. Side benefit: fibers
gain real C-stack overflow protection (RecursionError) on 3.14, which
they currently lack entirely (`boost::fibers::fixedsize_stack` has no
guard pages). Also makes `FiberState::kStackSize` public so the
anchoring uses the real fiber stack size.

## Related issue number

Closes #63290. Supersedes #63284 (same diagnosis direction, but
hand-rolled `_PyThreadStateImpl` offsets, a deliberate
`gilstate_counter` leak that freezes non-main threads, and a crash
premise that CPython 3.14.2 already fixed upstream).

## Checks

- Verified with a locally built cp314 Linux (aarch64, python:3.14.6
docker) wheel:
- refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks)
- `__del__` deferral probe: dealloc during return serialization on the
fiber **deferred → immediate**
- live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3
KiB/task** across async call → dict/bytes, async generator, sync
generator on async actor
- reporter-shaped streaming workload (400 tasks, 10 concurrent
sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped
regions 0 → 0
- async-actor smoke: correctness (echo, state, async generators,
recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s)
- throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs
baseline, same container image): fixed 5624–6076 tasks/s vs unpatched
4551–4825 tasks/s meaning no regression (the unpatched build is slower
while leaking)
- baseline (unpatched) wheel from the same tree reproduces the bug:
+4.00 refs/task, fiber dealloc deferred=True

note: fable did a majority of the heavy lifting in this investigation
with prompting on what to check next and validate the solution

---------

Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: myan <myan@anyscale.com>
Co-authored-by: Mark Towers <mark@anyscale.com>
Co-authored-by: myan <myan@anyscale.com>
Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com>

#65177)

…protection to fiber stacks (#64772)

# Description

On Python 3.14 + Linux, every async-actor task permanently leaks ~518
KiB of live malloc (the per-task `asyncio.Task`,
`concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack
`Packer`s with 256 KiB internal buffers).
Closes #63290

### Root cause

**1. CPython 3.14 changed how it avoids stack overflow when freeing
objects.** Freeing one object can recursively free many others (a dict
frees its values, which free their contents, …), and each level is a
nested C call. To keep that from overflowing the C stack, CPython has
long had a safety mechanism (the "trashcan"): when it decides it's too
deep, it doesn't free the object right away. Instead it parks the object
on a per-thread *delete-later* list and drains the list once there's
stack headroom again. Up to 3.13, "too deep" was a simple recursion
counter. In 3.14 it's decided by comparing the actual machine **stack
pointer** against the stack bounds CPython recorded for the thread when
it attached (from pthreads, on Linux).

**2. Ray async actors don't run task code on the thread's normal
stack.** Each task executes on a small 256 KiB boost fiber stack
allocated elsewhere in memory. The problem is that CPython still thinks
the thread runs on its original pthread stack.

So while a task runs on a fiber, every "am I near the stack limit?"
check compares the fiber's stack pointer against the *pthread* stack's
bounds. On Linux, fiber stacks happen to be allocated at lower addresses
than the pthread stack, so CPython concludes the stack is hopelessly
overflowed and parks **every** object freed during the task (including
return-value serialization and end-of-task cleanup) on the delete-later
list.

That list is only ever drained by a later free on the same thread state
at a healthy stack margin, which never happens here as the Ray thread
only runs on fibers and Ray creates a fresh Python thread state per task
and destroys it at task end. This means that CPython destroys a thread
state **without draining its delete-later list** and the parked objects
are orphaned permanently. That's the leak.

Why the confusing symptoms:

- `boost::make_fcontext` in the issue's flamegraphs just marks *where*
the leaked allocations were made (on a fiber stack); the fiber stacks
themselves are freed correctly.
- macOS is unaffected only by luck: fiber stacks there land at *higher*
addresses than the pthread stack, so the check passes.
- 3.13 and earlier are unaffected because their trashcan uses the
counter, not the stack pointer.

### Fix

CPython 3.14.2 added an official API for exactly this situation:
`PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets
an embedder tell CPython "this thread is currently executing on *this*
stack." We call it with the fiber's stack bounds:

- at async-actor task entry in `task_execution_handler`, and
- whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers
share the thread state, so each must re-register its own stack).

With the bounds correct, the near-limit check returns to normal
behavior: objects are freed immediately, and the rare genuinely-deep
free is parked and then properly drained.

Implementation notes: the symbol is looked up via `dlsym`, so `_raylet`
still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a
more severe, since-fixed stack-check bug anyway, python/cpython#141944).
No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are
derived from the current stack pointer minus a conservative allowance
for stack already used, so the protection errs toward triggering
slightly early rather than missing an overflow. Side benefit: fibers
gain real C-stack overflow protection (RecursionError) on 3.14, which
they currently lack entirely (`boost::fibers::fixedsize_stack` has no
guard pages). Also makes `FiberState::kStackSize` public so the
anchoring uses the real fiber stack size.

## Related issue number

Closes #63290. Supersedes #63284 (same diagnosis direction, but
hand-rolled `_PyThreadStateImpl` offsets, a deliberate
`gilstate_counter` leak that freezes non-main threads, and a crash
premise that CPython 3.14.2 already fixed upstream).

## Checks

- Verified with a locally built cp314 Linux (aarch64, python:3.14.6
docker) wheel:
- refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks)
- `__del__` deferral probe: dealloc during return serialization on the
fiber **deferred → immediate**
- live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3
KiB/task** across async call → dict/bytes, async generator, sync
generator on async actor
- reporter-shaped streaming workload (400 tasks, 10 concurrent
sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped
regions 0 → 0
- async-actor smoke: correctness (echo, state, async generators,
recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s)
- throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs
baseline, same container image): fixed 5624–6076 tasks/s vs unpatched
4551–4825 tasks/s meaning no regression (the unpatched build is slower
while leaking)
- baseline (unpatched) wheel from the same tree reproduces the bug:
+4.00 refs/task, fiber dealloc deferred=True

note: fable did a majority of the heavy lifting in this investigation
with prompting on what to check next and validate the solution

---------






(cherry picked from commit 35591ba)

Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: myan <myan@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Co-authored-by: Mark Towers <mark.m.towers@gmail.com>
Co-authored-by: Mark Towers <mark@anyscale.com>
Co-authored-by: myan <myan@anyscale.com>
Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com>

…protection to fiber stacks (ray-project#64772)

# Description

On Python 3.14 + Linux, every async-actor task permanently leaks ~518
KiB of live malloc (the per-task `asyncio.Task`,
`concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack
`Packer`s with 256 KiB internal buffers).
Closes ray-project#63290

### Root cause

**1. CPython 3.14 changed how it avoids stack overflow when freeing
objects.** Freeing one object can recursively free many others (a dict
frees its values, which free their contents, …), and each level is a
nested C call. To keep that from overflowing the C stack, CPython has
long had a safety mechanism (the "trashcan"): when it decides it's too
deep, it doesn't free the object right away. Instead it parks the object
on a per-thread *delete-later* list and drains the list once there's
stack headroom again. Up to 3.13, "too deep" was a simple recursion
counter. In 3.14 it's decided by comparing the actual machine **stack
pointer** against the stack bounds CPython recorded for the thread when
it attached (from pthreads, on Linux).

**2. Ray async actors don't run task code on the thread's normal
stack.** Each task executes on a small 256 KiB boost fiber stack
allocated elsewhere in memory. The problem is that CPython still thinks
the thread runs on its original pthread stack.

So while a task runs on a fiber, every "am I near the stack limit?"
check compares the fiber's stack pointer against the *pthread* stack's
bounds. On Linux, fiber stacks happen to be allocated at lower addresses
than the pthread stack, so CPython concludes the stack is hopelessly
overflowed and parks **every** object freed during the task (including
return-value serialization and end-of-task cleanup) on the delete-later
list.

That list is only ever drained by a later free on the same thread state
at a healthy stack margin, which never happens here as the Ray thread
only runs on fibers and Ray creates a fresh Python thread state per task
and destroys it at task end. This means that CPython destroys a thread
state **without draining its delete-later list** and the parked objects
are orphaned permanently. That's the leak.

Why the confusing symptoms:

- `boost::make_fcontext` in the issue's flamegraphs just marks *where*
the leaked allocations were made (on a fiber stack); the fiber stacks
themselves are freed correctly.
- macOS is unaffected only by luck: fiber stacks there land at *higher*
addresses than the pthread stack, so the check passes.
- 3.13 and earlier are unaffected because their trashcan uses the
counter, not the stack pointer.

### Fix

CPython 3.14.2 added an official API for exactly this situation:
`PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets
an embedder tell CPython "this thread is currently executing on *this*
stack." We call it with the fiber's stack bounds:

- at async-actor task entry in `task_execution_handler`, and
- whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers
share the thread state, so each must re-register its own stack).

With the bounds correct, the near-limit check returns to normal
behavior: objects are freed immediately, and the rare genuinely-deep
free is parked and then properly drained.

Implementation notes: the symbol is looked up via `dlsym`, so `_raylet`
still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a
more severe, since-fixed stack-check bug anyway, python/cpython#141944).
No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are
derived from the current stack pointer minus a conservative allowance
for stack already used, so the protection errs toward triggering
slightly early rather than missing an overflow. Side benefit: fibers
gain real C-stack overflow protection (RecursionError) on 3.14, which
they currently lack entirely (`boost::fibers::fixedsize_stack` has no
guard pages). Also makes `FiberState::kStackSize` public so the
anchoring uses the real fiber stack size.

## Related issue number

Closes ray-project#63290. Supersedes ray-project#63284 (same diagnosis direction, but
hand-rolled `_PyThreadStateImpl` offsets, a deliberate
`gilstate_counter` leak that freezes non-main threads, and a crash
premise that CPython 3.14.2 already fixed upstream).

## Checks

- Verified with a locally built cp314 Linux (aarch64, python:3.14.6
docker) wheel:
- refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks)
- `__del__` deferral probe: dealloc during return serialization on the
fiber **deferred → immediate**
- live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3
KiB/task** across async call → dict/bytes, async generator, sync
generator on async actor
- reporter-shaped streaming workload (400 tasks, 10 concurrent
sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped
regions 0 → 0
- async-actor smoke: correctness (echo, state, async generators,
recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s)
- throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs
baseline, same container image): fixed 5624–6076 tasks/s vs unpatched
4551–4825 tasks/s meaning no regression (the unpatched build is slower
while leaking)
- baseline (unpatched) wheel from the same tree reproduces the bug:
+4.00 refs/task, fiber dealloc deferred=True

note: fable did a majority of the heavy lifting in this investigation
with prompting on what to check next and validate the solution

---------

Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: myan <myan@anyscale.com>
Co-authored-by: Mark Towers <mark@anyscale.com>
Co-authored-by: myan <myan@anyscale.com>
Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com>

…protection to fiber stacks (ray-project#64772)

# Description

On Python 3.14 + Linux, every async-actor task permanently leaks ~518
KiB of live malloc (the per-task `asyncio.Task`,
`concurrent.futures.Future`, Cython coroutine + scopes, and two msgpack
`Packer`s with 256 KiB internal buffers).
Closes ray-project#63290

### Root cause

**1. CPython 3.14 changed how it avoids stack overflow when freeing
objects.** Freeing one object can recursively free many others (a dict
frees its values, which free their contents, …), and each level is a
nested C call. To keep that from overflowing the C stack, CPython has
long had a safety mechanism (the "trashcan"): when it decides it's too
deep, it doesn't free the object right away. Instead it parks the object
on a per-thread *delete-later* list and drains the list once there's
stack headroom again. Up to 3.13, "too deep" was a simple recursion
counter. In 3.14 it's decided by comparing the actual machine **stack
pointer** against the stack bounds CPython recorded for the thread when
it attached (from pthreads, on Linux).

**2. Ray async actors don't run task code on the thread's normal
stack.** Each task executes on a small 256 KiB boost fiber stack
allocated elsewhere in memory. The problem is that CPython still thinks
the thread runs on its original pthread stack.

So while a task runs on a fiber, every "am I near the stack limit?"
check compares the fiber's stack pointer against the *pthread* stack's
bounds. On Linux, fiber stacks happen to be allocated at lower addresses
than the pthread stack, so CPython concludes the stack is hopelessly
overflowed and parks **every** object freed during the task (including
return-value serialization and end-of-task cleanup) on the delete-later
list.

That list is only ever drained by a later free on the same thread state
at a healthy stack margin, which never happens here as the Ray thread
only runs on fibers and Ray creates a fresh Python thread state per task
and destroys it at task end. This means that CPython destroys a thread
state **without draining its delete-later list** and the parked objects
are orphaned permanently. That's the leak.

Why the confusing symptoms:

- `boost::make_fcontext` in the issue's flamegraphs just marks *where*
the leaked allocations were made (on a fiber stack); the fiber stacks
themselves are freed correctly.
- macOS is unaffected only by luck: fiber stacks there land at *higher*
addresses than the pthread stack, so the check passes.
- 3.13 and earlier are unaffected because their trashcan uses the
counter, not the stack pointer.

### Fix

CPython 3.14.2 added an official API for exactly this situation:
`PyUnstable_ThreadState_SetStackProtection` (python/cpython#141661) lets
an embedder tell CPython "this thread is currently executing on *this*
stack." We call it with the fiber's stack bounds:

- at async-actor task entry in `task_execution_handler`, and
- whenever a fiber resumes after `YieldCurrentFiber` (concurrent fibers
share the thread state, so each must re-register its own stack).

With the bounds correct, the near-limit check returns to normal
behavior: objects are freed immediately, and the rare genuinely-deep
free is parked and then properly drained.

Implementation notes: the symbol is looked up via `dlsym`, so `_raylet`
still imports on 3.14.0/3.14.1 (fix skipped there; those releases have a
more severe, since-fixed stack-check bug anyway, python/cpython#141944).
No-op below 3.14 (preprocessor-gated) and on Windows. Stack bounds are
derived from the current stack pointer minus a conservative allowance
for stack already used, so the protection errs toward triggering
slightly early rather than missing an overflow. Side benefit: fibers
gain real C-stack overflow protection (RecursionError) on 3.14, which
they currently lack entirely (`boost::fibers::fixedsize_stack` has no
guard pages). Also makes `FiberState::kStackSize` public so the
anchoring uses the real fiber stack size.

## Related issue number

Closes ray-project#63290. Supersedes ray-project#63284 (same diagnosis direction, but
hand-rolled `_PyThreadStateImpl` offsets, a deliberate
`gilstate_counter` leak that freezes non-main threads, and a crash
premise that CPython 3.14.2 already fixed upstream).

## Checks

- Verified with a locally built cp314 Linux (aarch64, python:3.14.6
docker) wheel:
- refcount probe: **+4.00 refs/task → 0.00/task** (100 tasks)
- `__del__` deferral probe: dealloc during return serialization on the
fiber **deferred → immediate**
- live-malloc probe (`mallinfo2`, 300 tasks/shape): **~518 KiB/task → ~3
KiB/task** across async call → dict/bytes, async generator, sync
generator on async actor
- reporter-shaped streaming workload (400 tasks, 10 concurrent
sessions): live-malloc delta **0.2 MB total**, fiber-sized mapped
regions 0 → 0
- async-actor smoke: correctness (echo, state, async generators,
recursion), concurrency (20 overlapping 0.5 s sleeps in 0.51 s)
- throughput A/B (500 sequential echo tasks, 3 runs fixed / 2 runs
baseline, same container image): fixed 5624–6076 tasks/s vs unpatched
4551–4825 tasks/s meaning no regression (the unpatched build is slower
while leaking)
- baseline (unpatched) wheel from the same tree reproduces the bug:
+4.00 refs/task, fiber dealloc deferred=True

note: fable did a majority of the heavy lifting in this investigation
with prompting on what to check next and validate the solution

---------

Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: myan <myan@anyscale.com>
Co-authored-by: Mark Towers <mark@anyscale.com>
Co-authored-by: myan <myan@anyscale.com>
Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com>
Signed-off-by: 400Ping <jiekaichang@apache.org>