gh-114746: Avoid quadratic behavior in free-threaded GC - #114817
Conversation
Sorry, something went wrong.
The free-threaded build's GC implementation is non-generational, but was scheduled as if it were collecting a young generation leading to quadratic behavior. This increases the minimum threshold and scales it to the number of live objects as we do for the old generation in the default build. Note that the scheduling is still not thread-safe without the GIL. Those changes will come in later PRs. A few tests, like "test_sneaky_frame_object" rely on prompt scheduling of the GC. For now, to keep that test passing, we disable the scaled threshold after calls like `gc.set_threshold(1, 0, 0)`.
|
!buildbot nogil |
Sorry, something went wrong.
|
🤖 New build scheduled with the buildbot fleet by @colesbury for commit 66959ca 🤖 The command will test the builders whose names match following regular expression: nogil The builders matched are:
|
Sorry, something went wrong.
|
@itamaro - I think this fixes the nogil Windows build bot timeouts. The "Ubuntu NoGIL Refleaks" buildbot still seems quite slow, but I think that was the case before the GC changes. |
Sorry, something went wrong.
|
Ubuntu refleaks failed with a transient issue, which looks unrelated: ======================================================================
ERROR: test_lockf (__main__.FNTLEINTRTest.test_lockf)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 532, in test_lockf
self._lock(fcntl.lockf, "lockf")
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/buildarea/pull_request.itamaro-ubuntu-aws.refleak.nogil/build/Lib/test/_test_eintr.py", line 517, in _lock
raise Exception("failed to sync child in %.1f sec" % dt)
Exception: failed to sync child in 300.7 sec
A re-run passed. But I'm sure the flock issue will come up again, especially when I get around to monitoring re-ran Buildbot tests. |
Sorry, something went wrong.
|
Thank you for the quick fix! |
Sorry, something went wrong.
…GH-114817) The free-threaded build's GC implementation is non-generational, but was scheduled as if it were collecting a young generation leading to quadratic behavior. This increases the minimum threshold and scales it to the number of live objects as we do for the old generation in the default build. Note that the scheduling is still not thread-safe without the GIL. Those changes will come in later PRs. A few tests, like "test_sneaky_frame_object" rely on prompt scheduling of the GC. For now, to keep that test passing, we disable the scaled threshold after calls like `gc.set_threshold(1, 0, 0)`.
The free-threaded build's GC implementation is non-generational, but was scheduled as if it were collecting a young generation leading to quadratic behavior. This increases the minimum threshold and scales it to the number of live objects as we do for the old generation in the default build.
Note that the scheduling is still not thread-safe without the GIL. Those changes will come in later PRs.
A few tests, like "test_sneaky_frame_object" rely on prompt scheduling of the GC. For now, to keep that test passing, we disable the scaled threshold after calls like gc.set_threshold(1, 0, 0).