FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-75876: Run bigmem tests in a subprocess by serhiy-storchaka · Pull Request #155302 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-75876: Run bigmem tests in a subprocess - #155302

Merged
serhiy-storchaka merged 4 commits into
python:mainfrom
serhiy-storchaka:bigmemtest-isolate
Aug 18, 2026
ConversationCommits (4)ChecksFiles changed
Merged

gh-75876: Run bigmem tests in a subprocess#155302
serhiy-storchaka merged 4 commits into
python:mainfrom
serhiy-storchaka:bigmemtest-isolate

Conversation

serhiy-storchaka commented Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

A test which really allocates the memory it asks for (that is, run with -M) now runs in a subprocess, so that the memory it uses and the address space it fragments are released when it ends. A dummy run stays in the process.

The parent process watches the memory usage of the subprocess while waiting for it, so Lib/test/memory_watchdog.py and the separate watchdog process are no longer needed. _SubprocessTest.wait() calls back every second between communicate() timeouts, which does not lose the subprocess output.

Tested with -M 4G and -M 20G on Linux, and with -M 4G on FreeBSD and on a Windows debug build.

pythongh-75876: Run bigmem tests in a subprocess
3a0c86a 
A test which really allocates the memory it asks for (that is, run with -M)
now runs in a subprocess, so that the memory it uses and the address space
it fragments are released when it ends.  A dummy run stays in the process.

The parent process watches the memory usage of the subprocess while waiting
for it, so the separate watchdog process is no longer needed.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
bedevere-app Bot mentioned this pull request Aug 6, 2026
Open
serhiy-storchaka added skip news tests Tests in the Lib/test dir labels Aug 6, 2026
serhiy-storchaka added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Aug 6, 2026
serhiy-storchaka enabled auto-merge (squash) August 18, 2026 07:08
serhiy-storchaka enabled auto-merge (squash) August 18, 2026 07:18
serhiy-storchaka merged commit 66d7c89 into python:main Aug 18, 2026
51 checks passed

miss-islington-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

miss-islington-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.15 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 66d7c891044915ccb7a12e0cc2c78991508e816e 3.15

miss-islington-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 66d7c891044915ccb7a12e0cc2c78991508e816e 3.14

miss-islington-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 66d7c891044915ccb7a12e0cc2c78991508e816e 3.13

bedevere-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

GH-156013 is a backport of this pull request to the 3.14 branch.

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Aug 18, 2026

bedevere-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

GH-156014 is a backport of this pull request to the 3.13 branch.

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Aug 18, 2026

BHUVANSH855 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@serhiy-storchaka,
If this PR still needs a backport to 3.15, should I proceed with?

serhiy-storchaka commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Backporting to 3.15 needs merging several other backports first.

serhiy-storchaka deleted the bigmemtest-isolate branch August 21, 2026 13:49
Yhg1s pushed a commit that referenced this pull request Sep 29, 2026
[3.13] gh-75876: Run bigmem tests in a subprocess (GH-155302) (#156014)
20c0747 
A test which really allocates the memory it asks for (that is, run with -M)
now runs in a subprocess, so that the memory it uses and the address space
it fragments are released when it ends.  A dummy run stays in the process.

The separate watchdog process is no longer used.  Unlike in the main branch,
the parent process does not report the memory usage of the subprocess:
get_process_memory_usage() (gh-150114) is not available on this branch.

(cherry picked from commit 66d7c89)
serhiy-storchaka added needs backport to 3.15 bugs and security fixes and removed needs backport to 3.15 bugs and security fixes labels Oct 9, 2026

miss-islington-app Bot commented Oct 9, 2026

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

miss-islington-app Bot commented Oct 9, 2026

Copy link
Copy Markdown

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.15 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker 66d7c891044915ccb7a12e0cc2c78991508e816e 3.15

serhiy-storchaka added a commit that referenced this pull request Oct 9, 2026
[3.14] gh-75876: Run bigmem tests in a subprocess (GH-155302) (GH-156013
2a76adb 
)

A test which really allocates the memory it asks for (that is, run with -M)
now runs in a subprocess, so that the memory it uses and the address space
it fragments are released when it ends.  A dummy run stays in the process.

The separate watchdog process is no longer used.  Unlike in the main branch,
the parent process does not report the memory usage of the subprocess:
get_process_memory_usage() (gh-150114) is not available on this branch.

(cherry picked from commit 66d7c89)

bedevere-app Bot commented Oct 9, 2026

Copy link
Copy Markdown

GH-159087 is a backport of this pull request to the 3.15 branch.

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 9, 2026
serhiy-storchaka added a commit that referenced this pull request Oct 10, 2026
[3.15] gh-75876: Run bigmem tests in a subprocess (GH-155302) (GH-159087
26fa570 
)

A test which really allocates the memory it asks for (that is, run with -M)
now runs in a subprocess, so that the memory it uses and the address space
it fragments are released when it ends.  A dummy run stays in the process.

The separate watchdog process is no longer used.  Unlike in the main branch,
the parent process does not report the memory usage of the subprocess:
get_process_memory_usage() (gh-150114) is not available on this branch.

(cherry picked from commit 66d7c89)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

vstinner Awaiting requested review from vstinner

Assignees

serhiy-storchaka

Labels

tests Tests in the Lib/test dir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL