FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix(core): close owned event loop on Zeroconf.close() to stop FD leak by bluetoothbot · Pull Request #1685 · python-zeroconf/python-zeroconf · GitHub

fix(core): close owned event loop on Zeroconf.close() to stop FD leak - #1685

Merged
bdraco merged 4 commits into
python-zeroconf:masterfrom
bluetoothbot:koan/fix-fd-leak-on-close
May 16, 2026
Merged

fix(core): close owned event loop on Zeroconf.close() to stop FD leak#1685
bdraco merged 4 commits into
python-zeroconf:masterfrom
bluetoothbot:koan/fix-fd-leak-on-close

Conversation

bluetoothbot commented May 16, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

What

Stop leaking file descriptors when a process repeatedly constructs and closes Zeroconf instances.

Why

Closes #1589. Each Zeroconf() built outside an existing asyncio loop spawns a private loop in a daemon thread. _shutdown_threads() previously called loop.stop() (via shutdown_loop) but never loop.close(), so the loop's selector (epoll FD on Linux) and its self-pipe sockets stayed open. Reproducer from the issue (20 cycles, before/after):

  • before: +30 FDs per 20 cycles (epoll + 2 sockets each)
  • after: 0 FDs per 50 cycles

Long-running services that recycle their socket set (the reflector use case the reporter described) eventually hit OSError: [Errno 24] Too many open files.

How

  • Call loop.close() after the loop thread joins in Zeroconf._shutdown_threads(). Only that thread-owned path needs it — when AsyncZeroconf reuses the caller's running loop we still leave it alone.
  • Short-circuit _shutdown_threads() on an already-closed loop so the documented idempotency of Zeroconf.close() is preserved (otherwise the second call's notify_all() raises RuntimeError: Event loop is closed).
  • Guard ServiceBrowser.cancel() against scheduling on a closed loop with the same rationale — Zeroconf.close() has already detached the browser via remove_all_service_listeners(), so the deferred _async_cancel had nothing left to do anyway.

Testing

  • New tests/test_core.py::Framework::test_close_releases_owned_event_loop — portable, asserts loop.is_closed() after close().
  • New tests/test_core.py::Framework::test_close_does_not_leak_file_descriptors — Linux-only, runs 20 cycles and asserts no per-cycle FD growth; both new tests fail on master.
  • Full suite: poetry run pytest tests/ --no-cov --timeout=60 -q → 333 passed, 2 skipped on Python 3.12.
  • poetry run ruff check / ruff format --check clean on the three touched files.

Quality Report

Changes: 3 files changed, 50 insertions(+), 2 deletions(-)

Code scan: clean

Tests: failed (timeout (120s))

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

codspeed-hq Bot commented May 16, 2026
edited
Loading

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing bluetoothbot:koan/fix-fd-leak-on-close (7da73d8) with master (277f80d)

codecov Bot commented May 16, 2026
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.76%. Comparing base (277f80d) to head (7da73d8).
⚠️ Report is 101 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1685   +/-   ##
=======================================
  Coverage   99.76%   99.76%           
=======================================
  Files          33       33           
  Lines        3401     3407    +6     
  Branches      461      463    +2     
=======================================
+ Hits         3393     3399    +6     
  Misses          5        5           
  Partials        3        3           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Pull request overview

Fixes a file-descriptor leak when repeatedly constructing and closing Zeroconf() instances that spin up their own background asyncio event loop, by ensuring the owned loop is properly closed during shutdown.

Changes:

  • Close the internally-owned asyncio event loop after its thread joins to release selector/self-pipe FDs.
  • Preserve Zeroconf.close() idempotency by short-circuiting shutdown when the loop is already closed.
  • Add regression tests for loop closure and (Linux-only) FD leak prevention; guard ServiceBrowser.cancel() against scheduling work onto a closed loop.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/zeroconf/_core.py Close owned event loop on shutdown and avoid errors on repeated close() calls.
src/zeroconf/_services/browser.py Prevent ServiceBrowser.cancel() from scheduling onto a closed loop.
tests/test_core.py Add regression tests for owned-loop closure and Linux FD leak detection.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/zeroconf/_services/browser.py Outdated

bdraco commented May 16, 2026

Copy link
Copy Markdown
Member

@bluetoothbot rr

bluetoothbot commented May 16, 2026
edited
Loading

Copy link
Copy Markdown
Contributor Author

PR Review — fix(core): close owned event loop on Zeroconf.close() to stop FD leak

The fix itself is correct and well-scoped: closing the owned loop after the thread joins plugs the FD leak, the idempotency guard preserves the documented Zeroconf.close() contract, and the browser-cancel guard handles the close-after-cancel race. The new tests fail on master and pass on this branch, which is exactly the right shape for a regression test. Before merging, please address @bdraco's contextlib.suppress request in browser.py:803-806 and reword the misleading "nothing left to do" comment that the Copilot bot flagged. The other items (docstring trim, test threshold) are stylistic suggestions, not blockers.


🟢 Suggestions

1. Use contextlib.suppress per maintainer request (`src/zeroconf/_services/browser.py`, L803-806)

@bdraco requested with contextlib.suppress(RuntimeError): here. It's the project's preferred idiom for narrow exception-swallow blocks and reads more clearly than try/except: pass — the intent ("close-loop race is expected, ignore it") is right there in the with line. Suggested change:

with contextlib.suppress(RuntimeError):
    if not self.zc.loop.is_closed():
        self.zc.loop.call_soon_threadsafe(self._async_cancel)

Add import contextlib at the top of the module if it's not already imported.

try:
    if not self.zc.loop.is_closed():
        self.zc.loop.call_soon_threadsafe(self._async_cancel)
except RuntimeError:
    pass
2. Comment overstates what _async_cancel does (`src/zeroconf/_services/browser.py`, L796-802)

The comment claims _async_cancel had "nothing left to do anyway" once Zeroconf.close() detaches the browser. That's misleading — _async_cancel also stops the query scheduler and cancels the query-sender task, which is the normal cleanup path while the loop is running. The skip only happens when the loop is already closed, where scheduling is impossible regardless.

Suggested rewording, dropping the inaccurate "nothing to do" claim:

# Skip scheduling _async_cancel when the loop is already closed:
# call_soon_threadsafe would raise RuntimeError. The normal path
# (loop running) still uses _async_cancel to stop the query
# scheduler and cancel the query-sender task. is_closed() is racy
# against a concurrent Zeroconf.close(), so suppress RuntimeError
# from call_soon_threadsafe as well.

Per the project's docstring/comment guidelines, this is one of the cases where a comment is justified (subtle race, RFC-style invariant), so a precise wording matters.

3. Idempotency comment is redundant with the function contract (`src/zeroconf/_core.py`, L679-684)

Per CLAUDE.md's commenting bar, this 3-line comment restates what the early-return + the Zeroconf.close() docstring (one frame up the call stack) already say. A future reader looking at if self.loop.is_closed(): return will infer "second call is a no-op" from the code itself. Consider dropping the comment (or trimming to a single line, e.g. # Second close() — loop already torn down.). The self.loop.close() block at the bottom already has a useful comment that explains the why (selector + self-pipe FDs); that one is worth keeping.

if self.loop.is_closed():
    # close() is documented as idempotent — a second call after the
    # loop has been torn down must be a no-op rather than raising.
    return
4. Test docstrings retell production-side story (`tests/test_core.py`, L89-94)

CLAUDE.md explicitly calls this out: "A test docstring should name what the test pins, in one sentence — not re-explain the bug, the fix, or the surrounding flow." The current docstrings describe the FD-leak mechanism (selector, self-pipe sockets, FD limit exhaustion) — that's PR-description / commit-message territory.

Suggested trims:

def test_close_releases_owned_event_loop(self):
    """Zeroconf.close() closes the event loop it started itself."""

def test_close_does_not_leak_file_descriptors(self):
    """Repeated Zeroconf()/close() does not grow the process FD table."""

The issue-number cross-reference also belongs in the PR body, not the test docstring (per the same section of CLAUDE.md).

5. FD-leak threshold could be tighter (`tests/test_core.py`, L117-121)

The threshold _fd_count() - baseline < 10 over 10 cycles allows ~1 FD growth per cycle, which is a lot of slack relative to the bug (~3 FDs per cycle = 30 over 10). If coverage/test harness FDs are the concern, the warm-up cycle already absorbs most one-shots before baseline is captured. A tighter bound like < 5 would still tolerate harness noise but would catch a partial regression (e.g. someone reintroduces just the epoll FD leak but not the self-pipe pair). Optional — current threshold does catch the original bug.

assert _fd_count() - baseline < 10

Checklist

  • Resource cleanup in error paths (FDs, sockets, loop)
  • Idempotency of public close()
  • Race-condition handling (cancel vs. close)
  • No bare except / no silent error swallowing
  • Test isolation (per-pid /proc, warm-up cycle)
  • Test docstrings describe behaviour, not implementation — suggestion #4
  • Comments justified per project bar — suggestion #2, #3
  • Skip guards for non-Linux / restricted-container CI
  • Cython .pxd updates not required (no cdef-class layout change)
  • Conventional Commits subject (fix(core): …)
  • No Co-Authored-By trailers from automated agents

Summary

The fix itself is correct and well-scoped: closing the owned loop after the thread joins plugs the FD leak, the idempotency guard preserves the documented Zeroconf.close() contract, and the browser-cancel guard handles the close-after-cancel race. The new tests fail on master and pass on this branch, which is exactly the right shape for a regression test. Before merging, please address @bdraco's contextlib.suppress request in browser.py:803-806 and reword the misleading "nothing left to do" comment that the Copilot bot flagged. The other items (docstring trim, test threshold) are stylistic suggestions, not blockers.


Automated review by Kōan8390d25
4351166
4aa37f7

Each Zeroconf() instance constructed without an active asyncio loop
spins up a private loop in a daemon thread. _shutdown_threads()
stopped that loop but never called loop.close(), so the selector
(epoll fd on Linux) and the loop's self-pipe sockets stayed open
for the lifetime of the process. Tight construct/close cycles
(e.g. an mDNS reflector that recycles its socket set) exhausted the
process FD limit.

Close the loop after the thread joins, and guard _shutdown_threads()
+ ServiceBrowser.cancel() against a re-entry on the now-closed loop
so the documented "close is idempotent" contract still holds.

Closes python-zeroconf#1589.

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/fix-fd-leak-on-close was rebased onto master and review feedback was applied.

Changes applied

  • Reworded comment in src/zeroconf/_services/browser.py per Copilot reviewer.
  • Comment now states we skip call_soon_threadsafe because the loop is closed (not "nothing to do"), and notes that _async_cancel normally stops the query scheduler and cancels the query-sender task while the loop runs.

Stats

3 files changed, 51 insertions(+), 2 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No recent commit on master addresses FD leak from unclosed event loop in Zeroconf.close().)
  • Rebased koan/fix-fd-leak-on-close onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run passed
  • Force-pushed koan/fix-fd-leak-on-close to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comment thread tests/test_core.py
Comment thread tests/test_core.py Outdated
Comment thread src/zeroconf/_services/browser.py Outdated

bdraco commented May 16, 2026

Copy link
Copy Markdown
Member

@bluetoothbot rr

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/fix-fd-leak-on-close was rebased onto master and review feedback was applied.

Changes applied

  • Done.
  • Summary

  • src/zeroconf/_services/browser.py — Reworded cancel() comment per Copilot review: emphasizes skip happens because loop closed, and that _async_cancel (stops query scheduler, cancels query-sender task) is normal cleanup path while loop runs. Wrapped call_soon_threadsafe in try/except RuntimeError to close TOCTOU race between is_closed() check and the call when cancel() runs concurrently with Zeroconf.close().
  • tests/test_core.py — Added second skipUnless guarding /proc/<pid>/fd directory existence per Copilot review (restricted containers may lack /proc mount). Reduced FD-leak loop from 20 to 10 cycles per Copilot review to cut CI runtime; threshold still rejects ~3-FDs-per-cycle regression (>=30 over 10 cycles).

Stats

3 files changed, 56 insertions(+), 2 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No commit on master addresses FD leak from event loop not being closed in Zeroconf._shutdown_threads)
  • Rebased koan/fix-fd-leak-on-close onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run passed
  • Force-pushed koan/fix-fd-leak-on-close to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

Comment thread src/zeroconf/_services/browser.py Outdated

bdraco commented May 16, 2026

Copy link
Copy Markdown
Member

@bluetoothbot rr

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/fix-fd-leak-on-close was rebased onto master and review feedback was applied.

Changes applied

  • Changes:
  • src/zeroconf/_services/browser.py: Replaced try/except RuntimeError: pass with contextlib.suppress(RuntimeError) per @bdraco and @bluetoothbot. Added import contextlib. Reworded comment per @Copilot to lead with what _async_cancel normally does and frame the skip as "solely because loop is closed" rather than "nothing left to do".
  • Other Copilot review points (FD-dir skip guard, iteration-count reduction) already present in prior commit (@unittest.skipUnless(Path(f"/proc/{os.getpid()}/fd").is_dir(), ...), range(10)).

Stats

3 files changed, 56 insertions(+), 2 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No recent master commits address FD leak on Zeroconf.close() or call loop.close() on owned event loo)
  • Rebased koan/fix-fd-leak-on-close onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run passed
  • Force-pushed koan/fix-fd-leak-on-close to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

bdraco marked this pull request as ready for review May 16, 2026 19:11
bdraco merged commit 2f78370 into python-zeroconf:master May 16, 2026
43 checks passed
bluetoothbot deleted the koan/fix-fd-leak-on-close branch May 16, 2026 20:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Socket leak when opening, and closing sockets

3 participants


Back | FazBrowse Home | New Git URL