FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping by aidaodedjl · Pull Request #156345 · python/cpython · GitHub

/ cpython Public

gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping - #156345

Open
aidaodedjl wants to merge 3 commits into
python:mainfrom
aidaodedjl:gh-156344-selector-self-pipe-eof
Open

gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping#156345
aidaodedjl wants to merge 3 commits into
python:mainfrom
aidaodedjl:gh-156344-selector-self-pipe-eof

Conversation

Copy link
Copy Markdown

Root cause

BaseSelectorEventLoop wakes itself through a socketpair created in
_make_self_pipe(). When that pair reaches a clean EOF — on Windows the
pair is a loopback TCP connection, and the OS can tear it down across
power or session state changes (sleep/hibernate, RDP disconnect, fast
user switching) — _read_from_self() breaks out of its drain loop:

    if not data:
        return

but the reader stays registered on the dead _ssock. A peer-closed
socket is permanently readable, so every select() iteration re-fires
the callback: one core pinned at 100% CPU with nothing logged,
measured at 582,692 _read_from_self invocations during a 3-second
idle sleep
(1.23s CPU). The loop never recovers.

This is the selector-side twin of gh-156333 (proactor loop, fixed in
#156343); the reporter of that issue hit the same teardown on a
WindowsSelectorEventLoop at 3.13.15.

The fix

_rebuild_self_pipe(), called from _read_from_self() on clean EOF,
mirrors the proactor-side approach with selector-specific care:

  1. Allocate the replacement first. socket.socketpair(), both ends
    non-blocking. An allocation failure leaves the previous state intact.
  2. Move the process-wide signal wakeup fd — only if we own it.
    signal.set_wakeup_fd(-1) returns the previously registered fd; move
    the registration to the new _csock only when it was pointed at our
    old _csock. A foreign registration (another loop's self-pipe, a
    test fixture) is restored untouched via set_wakeup_fd(old_fd).
  3. Non-main-thread hardening. set_wakeup_fd only works on the
    main thread; a rebuild triggered from a worker thread keeps the old
    _csock open so signal delivery keeps working (one socket leaked
    beats silently breaking Python's signal handling), and skips the
    wakeup-fd move for the same reason.
  4. Swap atomically-ish. Remove the old reader, close the old
    sockets (old _csock closed only when it was safe to move), then
    register the reader on the new _ssock — restoring the
    cross-thread wakeup invariant _make_self_pipe() established.

ConnectionResetError still propagates to the caller unchanged — that
is sock_recv's documented contract and the loop's error-handling path
(call_exception_handler) remains the right place for it.

Tests

Six new tests in test_selector_events.py:

Test Verifies
test_read_from_self_eof_rebuilds_self_pipe clean EOF triggers rebuild: new pair swapped in, old reader removed, new reader registered
test_read_from_self_blocking_is_not_eof BlockingIOError does NOT trigger a rebuild (guard)
test_self_pipe_eof_rebuild_functional real loop, real sockets: _csock.shutdown() → rebuild fires exactly once, reader lands on the new fd, cross-thread call_soon_threadsafe still wakes the loop
test_rebuild_self_pipe_moves_wakeup_fd wakeup fd registered on our _csock is migrated to the new one
test_rebuild_self_pipe_leaves_foreign_wakeup_fd a foreign wakeup fd is restored untouched
test_rebuild_self_pipe_no_signals no signal handlers in use → set_wakeup_fd never called

All six fail on the pre-fix code (verified by checking out the original
selector_events.py); after the fix the full test_asyncio suite
passes on Windows (34/34 files, incl. test_selector_events and
test_windows_events) and on macOS.

Measured on the Windows repro from the issue: 582,692 callback
invocations / 1.23s CPU during 3s idle → 1 invocation (the rebuild
itself) / 0.00s CPU, with cross-thread wakeup intact.

…y-looping

When the self-pipe socketpair of a BaseSelectorEventLoop reaches a clean
EOF (e.g. the OS tears the connection down across a power or session
state change on Windows), _read_from_self broke out of its read loop but
left the reader registered on the dead socket.  A closed-for-read socket
is permanently readable, so every select() iteration re-fired the
callback: one core pinned at 100% CPU with nothing logged, measured at
582k callback invocations during a 3-second idle sleep.

Rebuild the pair instead: allocate the replacement before touching the
old sockets so an allocation failure leaves the previous state intact,
move the process-wide signal wakeup fd to the new socket when (and only
when) it is registered on our _csock -- restoring foreign registrations
untouched, and keeping the old write end open when it cannot be moved
from a worker thread -- then remove the old reader, close the old
sockets, and register the reader on the new socket.
aidaodedjl changed the title gh-156344: rebuild the selector self-pream EOF instead of busy-looping gh-156344: rebuild the selector self-pipe on EOF instead of busy-looping Aug 25, 2026

python-cla-bot Bot commented Aug 25, 2026
edited
Loading

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL