FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-159879: Do not limit the C stack below what the interpreter assumes · python/cpython@b65f636 · GitHub

Repository navigation

Commit b65f636

Browse files
gh-159879: Do not limit the C stack below what the interpreter assumes
Where a thread's stack size cannot be queried, the interpreter assumes Py_C_STACK_SIZE, and a smaller thread overflows instead of raising RecursionError. run_with_limited_c_stack() now raises such a size to C_STACK_SIZE. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 036d309 commit b65f636

2 files changed

Lines changed: 54 additions & 3 deletions

File tree

‎Lib/test/support/__init__.py‎

Lines changed: 49 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2940,11 +2940,46 @@ def skip_if_huge_c_stack(depth=150_000):
29402940

29412941

29422942
# Small enough to be exhausted by tens of thousands of recursive calls,
2943-
# but not smaller than Py_C_STACK_SIZE (4 MiB) which the interpreter
2944-
# assumes if it cannot query the thread stack size.
2943+
# but not smaller than Py_C_STACK_SIZE (up to 4 MiB, depending on the
2944+
# platform and on the stack size the build passes to the linker) which the
2945+
# interpreter assumes if it cannot query the thread stack size.
29452946
C_STACK_SIZE = 8 * 1024 * 1024
29462947

29472948

2949+
@functools.cache
2950+
def _c_stack_size_is_queryable():
2951+
"""Whether the interpreter can see the real C stack size of its thread.
2952+
2953+
On some platforms (currently AIX, NetBSD and non-glibc Linux) it cannot,
2954+
and assumes a fixed Py_C_STACK_SIZE instead. A thread created there with
2955+
a smaller stack overflows before the recursion guard notices, crashing
2956+
the interpreter rather than raising RecursionError.
2957+
"""
2958+
try:
2959+
from _testinternalcapi import get_c_recursion_remaining
2960+
except ImportError:
2961+
return False
2962+
import threading
2963+
remaining = []
2964+
def probe():
2965+
remaining.append(get_c_recursion_remaining())
2966+
old_size = threading.stack_size()
2967+
try:
2968+
# Both sizes are larger than any Py_C_STACK_SIZE, so probing is safe.
2969+
for size in (C_STACK_SIZE, 2 * C_STACK_SIZE):
2970+
threading.stack_size(size)
2971+
thread = threading.Thread(target=probe)
2972+
thread.start()
2973+
thread.join()
2974+
except (ValueError, RuntimeError):
2975+
return False
2976+
finally:
2977+
threading.stack_size(old_size)
2978+
# Threads of different sizes report the same headroom only if the
2979+
# interpreter is using its fixed estimate instead of the real size.
2980+
return len(remaining) == 2 and remaining[0] != remaining[1]
2981+
2982+
29482983
def run_with_limited_c_stack(depth=150_000, size=C_STACK_SIZE):
29492984
"""Decorator for tests exhausting the C stack with *depth* recursive calls.
29502985
@@ -2954,11 +2989,22 @@ def run_with_limited_c_stack(depth=150_000, size=C_STACK_SIZE):
29542989
29552990
If a thread with the limited C stack cannot be created, run the test in
29562991
the current thread, but skip it if the C stack is too large.
2992+
2993+
*size* is raised to C_STACK_SIZE on platforms where the interpreter
2994+
cannot query a thread's real C stack size, since a smaller thread would
2995+
crash there instead of raising RecursionError.
29572996
"""
29582997
reason = f"the C stack is large enough for {depth} recursive calls"
29592998
def decorator(test):
29602999
@functools.wraps(test)
29613000
def wrapper(*args, **kwargs):
3001+
thread_size = size
3002+
if thread_size < C_STACK_SIZE and not _c_stack_size_is_queryable():
3003+
# The interpreter would assume more stack than the thread
3004+
# really has and overflow it instead of raising
3005+
# RecursionError. Use the safe default size instead.
3006+
thread_size = C_STACK_SIZE
3007+
29623008
def run_test():
29633009
# The C stack can still be too large if limiting it failed.
29643010
if _has_huge_c_stack(depth):
@@ -2967,7 +3013,7 @@ def run_test():
29673013

29683014
try:
29693015
import threading
2970-
old_size = threading.stack_size(size)
3016+
old_size = threading.stack_size(thread_size)
29713017
except (ImportError, ValueError, RuntimeError):
29723018
# Setting the thread stack size is not supported.
29733019
return run_test()
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
``test.support.run_with_limited_c_stack()`` no longer creates a thread
2+
smaller than ``test.support.C_STACK_SIZE`` on platforms where the
3+
interpreter cannot query a thread's real C stack size and assumes
4+
``Py_C_STACK_SIZE`` instead. Such a thread overflowed its stack rather
5+
than raising :exc:`RecursionError`.

0 commit comments

Comments
 (0)

Back | FazBrowse Home | New Git URL