gh-109700: Improve stress tests for interpreter creation by serhiy-storchaka · Pull Request #109946 · python/cpython · GitHub

Ensure that destructors are called in the test that created interpreters, not after finishing it.
Try to create interpreters simultaneously in test_create_many_threaded.
Mark test_create_many_threaded that requires over 6GB of memory as bigmemtest.

Issue: test_interpreters: test_create_many_threaded() failed on FreeBSD: log: RuntimeError: interpreter creation failed #109700

…stressful

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 6aecbf9 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

While leaks are not yet fixed, we want to minimize effect of these tests on other tests.

Ensure that destructors are called in the test that created interpreters, not after finishing it.
Try to create/run interpreters in threads simultaneously.
Mark tests that requires over 6GB of memory with bigmemtest.

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

…nGH-109946) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest. (cherry picked from commit 61b50a9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

GH-133391 is a backport of this pull request to the 3.13 branch.

…09946) (GH-133391) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest. (cherry picked from commit 61b50a9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

* origin/main: (111 commits) pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385) pythongh-131178: Add tests for `ast` command-line interface (python#133329) Regenerate pcbuild.sln in Visual Studio 2022 (python#133394) pythongh-133042: disable HACL* HMAC on Emscripten (python#133064) pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387) pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946) pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388) pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830) pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368) pythongh-132457: make staticmethod and classmethod generic (python#132460) pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812) pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490) pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517) pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628) pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358) bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226) pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287) pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364) pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027) pythonGH-91048: Add utils for printing the call stack for asyncio tasks (python#133284) ...

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows10 3.13 (tier-1) has failed when building commit 5daeebb.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1501/builds/1058) and take a look at the build logs.
Check if the failure is related to this commit (5daeebb) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1501/builds/1058

Failed tests:

test_venv

Failed subtests:

test_paths_depend_on_site_initialization - test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 625, in _rmtree_unsafe
    os.unlink(fullname)
    ~~~~~~~~~^^^^^^^^^^
PermissionError: [WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line 92, in tearDown
    rmtree(self.env_dir)
    ~~~~~~^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 470, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 413, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 409, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 412, in _rmtree_inner
    _force_run(fullname, os.unlink, fullname)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_sysconfig.py", line 653, in test_paths_depend_on_site_initialization
    with self.venv() as venv:
         ~~~~~~~~~^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\venv.py", line 27, in from_tmpdir
    with tempfile.TemporaryDirectory(prefix=prefix, dir=dir, delete=delete) as tmpdir:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 950, in __exit__
    self.cleanup()
    ~~~~~~~~~~~~^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 954, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 934, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 790, in rmtree
    return _rmtree_unsafe(path, onexc)
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 629, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 909, in onexc
    _os.unlink(path)
    ~~~~~~~~~~^^^^^^
PermissionError: [WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line 92, in tearDown
    rmtree(self.env_dir)
    ~~~~~~^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 470, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 413, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 409, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 412, in _rmtree_inner
    _force_run(fullname, os.unlink, fullname)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'

…nGH-109946) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest.

WIP: pythongh-109700 : Make stress tests on interpreter creation more …

91af60c

…stressful

bedevere-app Bot added the tests Tests in the Lib/test dir label Sep 27, 2023

Use less threads.

18031c9

bedevere-app Bot mentioned this pull request Sep 27, 2023

test_interpreters: test_create_many_threaded() failed on FreeBSD: log: RuntimeError: interpreter creation failed #109700

Closed

serhiy-storchaka added 2 commits November 8, 2023 22:25

Merge branch 'main' into test_interpreters-test_create_many_threaded

ef226df

Mark test_create_many_threaded with bigmemtest.

6aecbf9

serhiy-storchaka changed the title WIP: gh-109700: Make stress tests on interpreter creation more stressful gh-109700: Improve stress tests on interpreter creation Nov 8, 2023

serhiy-storchaka added skip news needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes 🔨 test-with-buildbots Test PR w/ buildbots; report in status section and removed needs backport to 3.11 only security fixes labels Nov 8, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2023

serhiy-storchaka marked this pull request as ready for review November 8, 2023 21:35

bedevere-app Bot added the awaiting core review label Nov 8, 2023

serhiy-storchaka changed the title gh-109700: Improve stress tests on interpreter creation gh-109700: Improve stress tests for interpreter creation Nov 8, 2023

serhiy-storchaka added the needs backport to 3.13 bugs and security fixes label May 9, 2024

Yhg1s removed the needs backport to 3.12 only security fixes label Apr 8, 2025

serhiy-storchaka added 2 commits May 4, 2025 20:02

Merge branch 'main' into test_interpreters-test_create_many_threaded

fd7cf3c

Add guards also for other stress tests.

5dd48d6

serhiy-storchaka requested a review from ericsnowcurrently as a code owner May 4, 2025 17:49

Polishing.

b3cf0b3

serhiy-storchaka merged commit 61b50a9 into python:main May 4, 2025

bedevere-app Bot removed the awaiting core review label May 4, 2025

serhiy-storchaka deleted the test_interpreters-test_create_many_threaded branch May 4, 2025 18:53

bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label May 4, 2025

gh-109700: Improve stress tests for interpreter creation by serhiy-storchaka · Pull Request #109946 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-109700: Improve stress tests for interpreter creation - #109946

gh-109700: Improve stress tests for interpreter creation#109946
serhiy-storchaka merged 7 commits into
python:mainfrom
serhiy-storchaka:test_interpreters-test_create_many_threaded

serhiy-storchaka commented Sep 27, 2023 •
edited
Loading

Uh oh!

Uh oh!

bedevere-bot commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented May 4, 2025

Uh oh!

miss-islington-app Bot commented May 4, 2025

Uh oh!

bedevere-app Bot commented May 4, 2025

Uh oh!

bedevere-bot commented May 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-109700: Improve stress tests for interpreter creation by serhiy-storchaka · Pull Request #109946 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-109700: Improve stress tests for interpreter creation - #109946

gh-109700: Improve stress tests for interpreter creation#109946serhiy-storchaka merged 7 commits intopython:mainpython/cpython:mainfrom serhiy-storchaka:test_interpreters-test_create_many_threadedserhiy-storchaka/cpython:test_interpreters-test_create_many_threadedCopy head branch name to clipboard

Conversation

serhiy-storchaka commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented May 4, 2025

Uh oh!

miss-islington-app Bot commented May 4, 2025

Uh oh!

bedevere-app Bot commented May 4, 2025

Uh oh!

bedevere-bot commented May 4, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-109700: Improve stress tests for interpreter creation#109946
serhiy-storchaka merged 7 commits into
python:mainfrom
serhiy-storchaka:test_interpreters-test_create_many_threaded

serhiy-storchaka commented Sep 27, 2023 •
edited
Loading