gh-135335: flush stdout/stderr in forkserver after preloading modules by duaneg · Pull Request #135338 · python/cpython · GitHub

If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload.

Issue: Fork server doesn't flush stdout/stderr after preloading modules, potentially leaving buffered data to be inherited by child processes #135335

…odules If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload.

...or at least produce more useful output if they fail

… if the defaults change elsewhere in the future)

…use a not-completely-awful name for the module to test.

Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com>

LGTM

Co-authored-by: Victor Stinner <vstinner@python.org>

…nUqb_.rst Co-authored-by: Victor Stinner <vstinner@python.org>

LGTM. I just have a last suggestion.

LGTM

Thanks @duaneg for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

Thanks @duaneg for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

…odules (pythonGH-135338) If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload. (cherry picked from commit 9877d19) Co-authored-by: Duane Griffin <duaneg@dghda.com> Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

GH-135670 is a backport of this pull request to the 3.14 branch.

…odules (pythonGH-135338) If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload. (cherry picked from commit 9877d19) Co-authored-by: Duane Griffin <duaneg@dghda.com> Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

GH-135671 is a backport of this pull request to the 3.13 branch.

Thanks for the bug report and the fix @duaneg.

Thanks for the reviews, folks.

…modules (GH-135338) (#135671) gh-135335: flush stdout/stderr in forkserver after preloading modules (GH-135338) If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload. (cherry picked from commit 9877d19) Co-authored-by: Duane Griffin <duaneg@dghda.com> Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

…modules (GH-135338) (#135670) gh-135335: flush stdout/stderr in forkserver after preloading modules (GH-135338) If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload. (cherry picked from commit 9877d19) Co-authored-by: Duane Griffin <duaneg@dghda.com> Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

Thanks for thorough working on this PR!
Looking forward for the next one!

Thanks for thorough working on this PR! Looking forward for the next one!

Well, as it happens, I would love #132574 to get a review 🙂

It is a very simple bug and fix: an off-by-one error, that allows an out-of-bounds read, which affects basic string operations like replace(), and has been in our code base ~forever.

Ok, I'll take a look.

…odules (python#135338) If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload. Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

pythongh-135335 : flush stdout/stderr in forkserver after preloading m…

d59b229

…odules If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload.

duaneg requested a review from gpshead as a code owner June 10, 2025 09:45

bedevere-app Bot added the awaiting review label Jun 10, 2025

bedevere-app Bot mentioned this pull request Jun 10, 2025

Fork server doesn't flush stdout/stderr after preloading modules, potentially leaving buffered data to be inherited by child processes #135335

Closed

duaneg added 2 commits June 10, 2025 23:01

Attempt to fix MacOS test failures

552266f

...or at least produce more useful output if they fail

Check stderr first to get details of assertion failures

6e36eea

efimov-mikhail requested changes Jun 11, 2025

View reviewed changes

Comment thread Lib/test/mp_preload_flush.py Outdated

bedevere-app Bot added awaiting core review and removed awaiting review labels Jun 11, 2025

duaneg added 2 commits June 12, 2025 13:53

Further minor diagnostic improvements upon test failure

33f9956

Set start method explicitly to "forkserver" so it works on MacOS (and…

9944968

… if the defaults change elsewhere in the future)

duaneg requested a review from efimov-mikhail June 16, 2025 09:06

efimov-mikhail reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

vstinner reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

Address reviewer feedback: introduce some intermediate variables and …

1523e74

…use a not-completely-awful name for the module to test.

efimov-mikhail reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

efimov-mikhail reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py

duaneg and others added 2 commits June 17, 2025 23:27

Update Lib/test/_test_multiprocessing.py

2125574

Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com>

Additional review feedback: add comment and align ordering

db298cb

efimov-mikhail approved these changes Jun 17, 2025

View reviewed changes

vstinner reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

vstinner reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

vstinner reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/mp_preload_flush.py Outdated

duaneg and others added 2 commits June 18, 2025 00:44

Update Lib/test/mp_preload_flush.py

f9bab0a

Co-authored-by: Victor Stinner <vstinner@python.org>

vstinner reviewed Jun 17, 2025

View reviewed changes

Comment thread Lib/test/_test_multiprocessing.py Outdated

Comment thread Lib/test/_test_multiprocessing.py Outdated

Comment thread Misc/NEWS.d/next/Library/2025-06-10-21-42-04.gh-issue-135335.WnUqb_.rst Outdated

duaneg and others added 4 commits June 18, 2025 10:35

Update Lib/test/_test_multiprocessing.py

4d57d74

Co-authored-by: Victor Stinner <vstinner@python.org>

Update Misc/NEWS.d/next/Library/2025-06-10-21-42-04.gh-issue-135335.W…

e73ea34

…nUqb_.rst Co-authored-by: Victor Stinner <vstinner@python.org>

Remove redundant check

3f07ff7

Don't write an newline to avoid any implicit flushes that might happen

da4df44

vstinner approved these changes Jun 18, 2025

View reviewed changes

Comment thread Lib/test/mp_preload_flush.py Outdated

bedevere-app Bot added awaiting merge and removed awaiting core review labels Jun 18, 2025

Be consistent and do not rely on asserts being enabled

fdb30e1

efimov-mikhail approved these changes Jun 18, 2025

View reviewed changes

vstinner merged commit 9877d19 into python:main Jun 18, 2025

bedevere-app Bot removed the awaiting merge label Jun 18, 2025

vstinner added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Jun 18, 2025

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jun 18, 2025

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Jun 18, 2025

gh-135335: flush stdout/stderr in forkserver after preloading modules by duaneg · Pull Request #135338 · python/cpython · GitHub

Repository navigation

gh-135335: flush stdout/stderr in forkserver after preloading modules - #135338

gh-135335: flush stdout/stderr in forkserver after preloading modules#135338vstinner merged 15 commits intopython:mainpython/cpython:mainfrom duaneg:gh-135335duaneg/cpython:gh-135335Copy head branch name to clipboard

Conversation

duaneg commented Jun 10, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

efimov-mikhail left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

efimov-mikhail left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington-app Bot commented Jun 18, 2025

Uh oh!

miss-islington-app Bot commented Jun 18, 2025

Uh oh!

bedevere-app Bot commented Jun 18, 2025

Uh oh!

bedevere-app Bot commented Jun 18, 2025

Uh oh!

vstinner commented Jun 18, 2025

Uh oh!

duaneg commented Jun 18, 2025

Uh oh!

efimov-mikhail commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

duaneg commented Jun 18, 2025

Uh oh!

efimov-mikhail commented Jun 18, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-135335: flush stdout/stderr in forkserver after preloading modules#135338
vstinner merged 15 commits into
python:mainfrom
duaneg:gh-135335

duaneg commented Jun 10, 2025 •
edited by bedevere-app Bot
Loading

efimov-mikhail commented Jun 18, 2025 •
edited
Loading