bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe by izbyshev · Pull Request #11671 · python/cpython · GitHub

When used to run a new executable image, fork() is not a good choice
for process creation, especially if the parent has a large working set:
fork() needs to copy page tables, which is slow, and may fail on systems
where overcommit is disabled, despite that the child is not going to
touch most of its address space.

Currently, subprocess is capable of using posix_spawn() instead, which
normally provides much better performance. However, posix_spawn() does not
support many of child setup operations exposed by subprocess.Popen().
Most notably, it's not possible to express close_fds=True, which
happens to be the default, via posix_spawn(). As a result, most users
can't benefit from faster process creation, at least not without
changing their code.

However, Linux provides vfork() system call, which creates a new process
without copying the address space of the parent, and which is actually
used by C libraries to efficiently implement posix_spawn(). Due to sharing
of the address space and even the stack with the parent, extreme care
is required to use vfork(). At least the following restrictions must hold:

No signal handlers must execute in the child process. Otherwise, they
might clobber memory shared with the parent, potentially confusing it.
Any library function called after vfork() in the child must be
async-signal-safe (as for fork()), but it must also not interact with any
library state in a way that might break due to address space sharing
and/or lack of any preparations performed by libraries on normal fork().
POSIX.1 permits to call only execve() and _exit(), and later revisions
remove vfork() specification entirely. In practice, however, almost all
operations needed by subprocess.Popen() can be safely implemented on
Linux.
Due to sharing of the stack with the parent, the child must be careful
not to clobber local variables that are alive across vfork() call.
Compilers are normally aware of this and take extra care with vfork()
(and setjmp(), which has a similar problem).
In case the parent is privileged, special attention must be paid to vfork()
use, because sharing an address space across different privilege domains
is insecure[1].

This patch adds support for using vfork() instead of fork() on Linux
when it's possible to do safely given the above. In particular:

vfork() is not used if credential switch is requested. The reverse case
(simple subprocess.Popen() but another application thread switches
credentials concurrently) is not possible for pure-Python apps because
subprocess.Popen() and functions like os.setuid() are mutually excluded
via GIL. We might also consider to add a way to opt-out of vfork() (and
posix_spawn() on platforms where it might be implemented via vfork()) in
a future PR.
vfork() is not used if preexec_fn != None.

With this change, subprocess will still use posix_spawn() if possible, but
will fallback to vfork() on Linux in most cases, and, failing that,
to fork().

[1] https://ewontfix.com/7

https://bugs.python.org/issue35823

overall I think this PR is looking good.

When used to run a new executable image, fork() is not a good choice for process creation, especially if the parent has a large working set: fork() needs to copy page tables, which is slow, and may fail on systems where overcommit is disabled, despite that the child is not going to touch most of its address space. Currently, subprocess is capable of using posix_spawn() instead, which normally provides much better performance. However, posix_spawn() does not support many of child setup operations exposed by subprocess.Popen(). Most notably, it's not possible to express `close_fds=True`, which happens to be the default, via posix_spawn(). As a result, most users can't benefit from faster process creation, at least not without changing their code. However, Linux provides vfork() system call, which creates a new process without copying the address space of the parent, and which is actually used by C libraries to efficiently implement posix_spawn(). Due to sharing of the address space and even the stack with the parent, extreme care is required to use vfork(). At least the following restrictions must hold: * No signal handlers must execute in the child process. Otherwise, they might clobber memory shared with the parent, potentially confusing it. * Any library function called after vfork() in the child must be async-signal-safe (as for fork()), but it must also not interact with any library state in a way that might break due to address space sharing and/or lack of any preparations performed by libraries on normal fork(). POSIX.1 permits to call only execve() and _exit(), and later revisions remove vfork() specification entirely. In practice, however, almost all operations needed by subprocess.Popen() can be safely implemented on Linux. * Due to sharing of the stack with the parent, the child must be careful not to clobber local variables that are alive across vfork() call. Compilers are normally aware of this and take extra care with vfork() (and setjmp(), which has a similar problem). * In case the parent is privileged, special attention must be paid to vfork() use, because sharing an address space across different privilege domains is insecure[1]. This patch adds support for using vfork() instead of fork() on Linux when it's possible to do safely given the above. In particular: * vfork() is not used if credential switch is requested. The reverse case (simple subprocess.Popen() but another application thread switches credentials concurrently) is not possible for pure-Python apps because subprocess.Popen() and functions like os.setuid() are mutually excluded via GIL. We might also consider to add a way to opt-out of vfork() (and posix_spawn() on platforms where it might be implemented via vfork()) in a future PR. * vfork() is not used if `preexec_fn != None`. With this change, subprocess will still use posix_spawn() if possible, but will fallback to vfork() on Linux in most cases, and, failing that, to fork(). [1] https://ewontfix.com/7

@gpshead Would you mind to take a look?

It isn't clear if setsid() is safe after vfork(); better safe than sorry. Also adds a couple comments and some debug build assertions.

Thanks for this work! Waiting on CI and doing some test runs myself. I expect to merge it shortly if that goes well.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO 3.x has failed when building commit 976da90.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/402/builds/230) and take a look at the build logs.
Check if the failure is related to this commit (976da90) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/402/builds/230

Failed tests:

test_lib2to3

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 1�[K
remote: Enumerating objects: 16, done.�[K
remote: Counting objects:   6% (1/16)�[K
remote: Counting objects:  12% (2/16)�[K
remote: Counting objects:  18% (3/16)�[K
remote: Counting objects:  25% (4/16)�[K
remote: Counting objects:  31% (5/16)�[K
remote: Counting objects:  37% (6/16)�[K
remote: Counting objects:  43% (7/16)�[K
remote: Counting objects:  50% (8/16)�[K
remote: Counting objects:  56% (9/16)�[K
remote: Counting objects:  62% (10/16)�[K
remote: Counting objects:  68% (11/16)�[K
remote: Counting objects:  75% (12/16)�[K
remote: Counting objects:  81% (13/16)�[K
remote: Counting objects:  87% (14/16)�[K
remote: Counting objects:  93% (15/16)�[K
remote: Counting objects: 100% (16/16)�[K
remote: Counting objects: 100% (16/16), done.�[K
remote: Compressing objects:   8% (1/12)�[K
remote: Compressing objects:  16% (2/12)�[K
remote: Compressing objects:  25% (3/12)�[K
remote: Compressing objects:  33% (4/12)�[K
remote: Compressing objects:  41% (5/12)�[K
remote: Compressing objects:  50% (6/12)�[K
remote: Compressing objects:  58% (7/12)�[K
remote: Compressing objects:  66% (8/12)�[K
remote: Compressing objects:  75% (9/12)�[K
remote: Compressing objects:  83% (10/12)�[K
remote: Compressing objects:  91% (11/12)�[K
remote: Compressing objects: 100% (12/12)�[K
remote: Compressing objects: 100% (12/12), done.�[K
remote: Total 17 (delta 6), reused 4 (delta 4), pack-reused 1�[K
From https://github.com/python/cpython
 * branch            master     -> FETCH_HEAD
Reset branch 'master'

/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
In file included from /home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Modules/_sha3/sha3module.c:113:0:
/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c: In function ‘_PySHA3_KeccakP1600_ExtractAndAddLanes’:
/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Modules/_sha3/kcp/KeccakP-1600-inplace32BI.c:421:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
         ((UINT32*)(output+lanePosition*8))[0] = ((UINT32*)(input+lanePosition*8))[0] ^ (*(const UINT32*)(laneAsBytes+0));
         ^
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: skipping incompatible /usr/local/lib/libgcc_s.so when searching for -lgcc_s

test_tix skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run
test_kqueue skipped -- test works only on BSD
test_ttk_guionly skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_flock (__main__.FNTLEINTRTest) ... ok
test_lockf (__main__.FNTLEINTRTest) ... ok
test_read (__main__.OSEINTRTest) ... ok
test_wait (__main__.OSEINTRTest) ... ok
test_wait3 (__main__.OSEINTRTest) ... ok
test_wait4 (__main__.OSEINTRTest) ... ok
test_waitpid (__main__.OSEINTRTest) ... ok
test_write (__main__.OSEINTRTest) ... ok
test_devpoll (__main__.SelectEINTRTest) ... skipped 'need select.devpoll'
test_epoll (__main__.SelectEINTRTest) ... ok
test_kqueue (__main__.SelectEINTRTest) ... skipped 'need select.kqueue'
test_poll (__main__.SelectEINTRTest) ... ok
test_select (__main__.SelectEINTRTest) ... ok
test_sigtimedwait (__main__.SignalEINTRTest) ... ok
test_sigwaitinfo (__main__.SignalEINTRTest) ... ok
test_accept (__main__.SocketEINTRTest) ... ok
test_open (__main__.SocketEINTRTest) ... ok
test_os_open (__main__.SocketEINTRTest) ... ok
test_recv (__main__.SocketEINTRTest) ... ok
test_recvmsg (__main__.SocketEINTRTest) ... ok
test_send (__main__.SocketEINTRTest) ... ok
test_sendall (__main__.SocketEINTRTest) ... ok
test_sendmsg (__main__.SocketEINTRTest) ... ok
test_sleep (__main__.TimeEINTRTest) ... ok

----------------------------------------------------------------------
Ran 24 tests in 8.325s

OK (skipped=2)
test_winsound skipped -- No module named 'winsound'
test_startfile skipped -- object <module 'os' from '/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/os.py'> has no attribute 'startfile'
test_winconsoleio skipped -- test only relevant on win32
test_ioctl skipped -- Unable to open /dev/tty
test_msilib skipped -- No module named '_msi'
test_winreg skipped -- No module named 'winreg'
test_devpoll skipped -- test works only on Solaris OS family
Fatal Python error: Segmentation fault

Current thread 0x000003ffbaff7700 (most recent call first):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/btm_matcher.py", line 125 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 414 in refactor_tree
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 371 in refactor_string
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 339 in refactor_file
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/tests/test_all_fixers.py", line 24 in test_all_project_files
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 549 in _callTestMethod
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 592 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 652 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/runner.py", line 176 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/support/__init__.py", line 959 in _run_suite
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/support/__init__.py", line 1082 in run_unittest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 211 in _test_module
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 236 in _runtest_inner2
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 272 in _runtest_inner
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 142 in _runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 195 in runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest_mp.py", line 81 in run_tests_worker
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 659 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 639 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 717 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/regrtest.py", line 43 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/regrtest.py", line 47 in <module>
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/runpy.py", line 87 in _run_code
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/runpy.py", line 197 in _run_module_as_main
test_tk skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_ossaudiodev skipped -- [Errno 2] No such file or directory: '/dev/dsp'
Fatal Python error: Segmentation fault

Current thread 0x000003ffaa1f7700 (most recent call first):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/btm_matcher.py", line 125 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 414 in refactor_tree
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 371 in refactor_string
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/refactor.py", line 339 in refactor_file
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/lib2to3/tests/test_all_fixers.py", line 24 in test_all_project_files
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 549 in _callTestMethod
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 592 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/case.py", line 652 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/unittest/runner.py", line 176 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/support/__init__.py", line 959 in _run_suite
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/support/__init__.py", line 1082 in run_unittest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 211 in _test_module
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 236 in _runtest_inner2
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 272 in _runtest_inner
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 155 in _runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/runtest.py", line 195 in runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 319 in rerun_failed_tests
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 696 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 639 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/libregrtest/main.py", line 717 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/test/__main__.py", line 2 in <module>
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/runpy.py", line 87 in _run_code
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/Lib/runpy.py", line 197 in _run_module_as_main
make: *** [buildbottest] Segmentation fault

Cannot open file '/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto/build/test-results.xml' for upload

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang Installed 3.x has failed when building commit 976da90.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/531/builds/211) and take a look at the build logs.
Check if the failure is related to this commit (976da90) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/531/builds/211

Failed tests:

test_lib2to3

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 16, done.        
remote: Counting objects:   6% (1/16)        
remote: Counting objects:  12% (2/16)        
remote: Counting objects:  18% (3/16)        
remote: Counting objects:  25% (4/16)        
remote: Counting objects:  31% (5/16)        
remote: Counting objects:  37% (6/16)        
remote: Counting objects:  43% (7/16)        
remote: Counting objects:  50% (8/16)        
remote: Counting objects:  56% (9/16)        
remote: Counting objects:  62% (10/16)        
remote: Counting objects:  68% (11/16)        
remote: Counting objects:  75% (12/16)        
remote: Counting objects:  81% (13/16)        
remote: Counting objects:  87% (14/16)        
remote: Counting objects:  93% (15/16)        
remote: Counting objects: 100% (16/16)        
remote: Counting objects: 100% (16/16), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 17 (delta 7), reused 5 (delta 5), pack-reused 1        
From https://github.com/python/cpython
 * branch                  master     -> FETCH_HEAD
Reset branch 'master'

Python/pytime.c:154:10: warning: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:207:14: warning: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
Python/pytime.c:392:10: warning: implicit conversion from 'long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:228:82: note: expanded from macro '_Py_InIntegralTypeRange'
#define _Py_InIntegralTypeRange(type, v) (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                                                              ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
./Include/pymath.h:221:124: note: expanded from macro '_Py_IntegralTypeMax'
#define _Py_IntegralTypeMax(type) ((_Py_IntegralTypeSigned(type)) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
./Modules/timemodule.c:116:13: warning: code will never be executed [-Wunreachable-code]
            PyErr_SetString(PyExc_OverflowError,
            ^~~~~~~~~~~~~~~
./Modules/_threadmodule.c:1587:19: warning: implicit conversion from '_PyTime_t' (aka 'long') to 'double' changes value from 9223372036854775 to 9223372036854776 [-Wimplicit-int-float-conversion]
    timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6;
                  ^~~~~~~~~~~~~~~~~~~~~~~~~ ~
1 warning generated.
1 warning generated.

  WARNING: The script easy_install-3.10 is installed in '/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip3 and pip3.10 are installed in '/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

test_tix skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_winsound skipped -- No module named 'winsound'
test_winreg skipped -- No module named 'winreg'
test_zipfile64 skipped -- test requires loads of disk-space bytes and a long time to run
test_devpoll skipped -- test works only on Solaris OS family
test_ossaudiodev skipped -- [Errno 2] No such file or directory: '/dev/dsp'
Fatal Python error: Segmentation fault

Current thread 0x000003ff8e978b20 (most recent call first):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/btm_matcher.py", line 125 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 414 in refactor_tree
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 371 in refactor_string
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 339 in refactor_file
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/tests/test_all_fixers.py", line 24 in test_all_project_files
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 549 in _callTestMethod
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 592 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 652 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/runner.py", line 176 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/support/__init__.py", line 959 in _run_suite
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/support/__init__.py", line 1082 in run_unittest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 211 in _test_module
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 236 in _runtest_inner2
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 272 in _runtest_inner
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 142 in _runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 195 in runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest_mp.py", line 81 in run_tests_worker
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 659 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 639 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 717 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/regrtest.py", line 43 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/regrtest.py", line 47 in <module>
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/runpy.py", line 87 in _run_code
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/runpy.py", line 197 in _run_module_as_main
test_nis skipped -- No module named 'nis'
test_check_c_globals skipped -- c-analyzer directory could not be found
test_asdl_parser skipped -- test irrelevant for an installed Python
test_tk skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_ioctl skipped -- Unable to open /dev/tty
test_readline skipped -- No module named 'readline'
test_clinic skipped -- clinic directory could not be found
test_kqueue skipped -- test works only on BSD
test_ttk_guionly skipped -- Tk unavailable due to TclError: no display name and no $DISPLAY environment variab [...]
test_gdb skipped -- test_gdb only works on source builds at the moment.
test_winconsoleio skipped -- test only relevant on win32
test_startfile skipped -- object <module 'os' from '/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/os.py'> has no attribute 'startfile'
test_flock (__main__.FNTLEINTRTest) ... ok
test_lockf (__main__.FNTLEINTRTest) ... ok
test_read (__main__.OSEINTRTest) ... ok
test_wait (__main__.OSEINTRTest) ... ok
test_wait3 (__main__.OSEINTRTest) ... ok
test_wait4 (__main__.OSEINTRTest) ... ok
test_waitpid (__main__.OSEINTRTest) ... ok
test_write (__main__.OSEINTRTest) ... ok
test_devpoll (__main__.SelectEINTRTest) ... skipped 'need select.devpoll'
test_epoll (__main__.SelectEINTRTest) ... ok
test_kqueue (__main__.SelectEINTRTest) ... skipped 'need select.kqueue'
test_poll (__main__.SelectEINTRTest) ... ok
test_select (__main__.SelectEINTRTest) ... ok
test_sigtimedwait (__main__.SignalEINTRTest) ... ok
test_sigwaitinfo (__main__.SignalEINTRTest) ... ok
test_accept (__main__.SocketEINTRTest) ... ok
test_open (__main__.SocketEINTRTest) ... ok
test_os_open (__main__.SocketEINTRTest) ... ok
test_recv (__main__.SocketEINTRTest) ... ok
test_recvmsg (__main__.SocketEINTRTest) ... ok
test_send (__main__.SocketEINTRTest) ... ok
test_sendall (__main__.SocketEINTRTest) ... ok
test_sendmsg (__main__.SocketEINTRTest) ... ok
test_sleep (__main__.TimeEINTRTest) ... ok

----------------------------------------------------------------------
Ran 24 tests in 6.604s

OK (skipped=2)
test_msilib skipped -- No module named 'msilib'
Fatal Python error: Segmentation fault

Current thread 0x000003ff812f8b20 (most recent call first):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/btm_matcher.py", line 125 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 414 in refactor_tree
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 371 in refactor_string
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/refactor.py", line 339 in refactor_file
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/lib2to3/tests/test_all_fixers.py", line 24 in test_all_project_files
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 549 in _callTestMethod
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 592 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/case.py", line 652 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 122 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/suite.py", line 84 in __call__
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/unittest/runner.py", line 176 in run
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/support/__init__.py", line 959 in _run_suite
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/support/__init__.py", line 1082 in run_unittest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 211 in _test_module
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 236 in _runtest_inner2
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 272 in _runtest_inner
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 155 in _runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 195 in runtest
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 319 in rerun_failed_tests
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 696 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 639 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/libregrtest/main.py", line 717 in main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/regrtest.py", line 43 in _main
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/test/regrtest.py", line 47 in <module>
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/runpy.py", line 87 in _run_code
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.10/runpy.py", line 197 in _run_module_as_main

…fe (pythonGH-11671) * bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe When used to run a new executable image, fork() is not a good choice for process creation, especially if the parent has a large working set: fork() needs to copy page tables, which is slow, and may fail on systems where overcommit is disabled, despite that the child is not going to touch most of its address space. Currently, subprocess is capable of using posix_spawn() instead, which normally provides much better performance. However, posix_spawn() does not support many of child setup operations exposed by subprocess.Popen(). Most notably, it's not possible to express `close_fds=True`, which happens to be the default, via posix_spawn(). As a result, most users can't benefit from faster process creation, at least not without changing their code. However, Linux provides vfork() system call, which creates a new process without copying the address space of the parent, and which is actually used by C libraries to efficiently implement posix_spawn(). Due to sharing of the address space and even the stack with the parent, extreme care is required to use vfork(). At least the following restrictions must hold: * No signal handlers must execute in the child process. Otherwise, they might clobber memory shared with the parent, potentially confusing it. * Any library function called after vfork() in the child must be async-signal-safe (as for fork()), but it must also not interact with any library state in a way that might break due to address space sharing and/or lack of any preparations performed by libraries on normal fork(). POSIX.1 permits to call only execve() and _exit(), and later revisions remove vfork() specification entirely. In practice, however, almost all operations needed by subprocess.Popen() can be safely implemented on Linux. * Due to sharing of the stack with the parent, the child must be careful not to clobber local variables that are alive across vfork() call. Compilers are normally aware of this and take extra care with vfork() (and setjmp(), which has a similar problem). * In case the parent is privileged, special attention must be paid to vfork() use, because sharing an address space across different privilege domains is insecure[1]. This patch adds support for using vfork() instead of fork() on Linux when it's possible to do safely given the above. In particular: * vfork() is not used if credential switch is requested. The reverse case (simple subprocess.Popen() but another application thread switches credentials concurrently) is not possible for pure-Python apps because subprocess.Popen() and functions like os.setuid() are mutually excluded via GIL. We might also consider to add a way to opt-out of vfork() (and posix_spawn() on platforms where it might be implemented via vfork()) in a future PR. * vfork() is not used if `preexec_fn != None`. With this change, subprocess will still use posix_spawn() if possible, but will fallback to vfork() on Linux in most cases, and, failing that, to fork(). [1] https://ewontfix.com/7 Co-authored-by: Gregory P. Smith [Google LLC] <gps@google.com>

By default python 3.10 will use the fork() which has to copy all the memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

By default python 3.10 will use the fork() which has to copy memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided and subprocess creation does not get discernibly slow the larger the Home Assistant python process grows. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

) * use posix spawn on alpine * Avoid subprocess memory copy when c library supports posix_spawn By default python 3.10 will use the fork() which has to copy all the memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn * Avoid subprocess memory copy when c library supports posix_spawn By default python 3.10 will use the fork() which has to copy memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided and subprocess creation does not get discernibly slow the larger the Home Assistant python process grows. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn * missed some * adjust more tests * coverage

…e-assistant#87958) * use posix spawn on alpine * Avoid subprocess memory copy when c library supports posix_spawn By default python 3.10 will use the fork() which has to copy all the memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn * Avoid subprocess memory copy when c library supports posix_spawn By default python 3.10 will use the fork() which has to copy memory of the parent process (in our case this can be huge since Home Assistant core can use hundreds of megabytes of RAM). By using posix_spawn this is avoided and subprocess creation does not get discernibly slow the larger the Home Assistant python process grows. In python 3.11 vfork will also be available python/cpython#80004 (comment) python/cpython#11671 but we won't always be able to use it and posix_spawn is considered safer https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 The subprocess library doesn't know about musl though even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c so we have to teach it since it only has checks for glibc https://github.com/python/cpython/blob/1b736838e6ae1b4ef42cdd27c2708face908f92c/Lib/subprocess.py#L745 The constant is documented as being able to be flipped here: https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn * missed some * adjust more tests * coverage

izbyshev requested a review from gpshead as a code owner January 25, 2019 02:07

the-knights-who-say-ni added the CLA signed label Jan 25, 2019

bedevere-bot added the awaiting review label Jan 25, 2019

izbyshev commented Jan 25, 2019

View reviewed changes

Comment thread Modules/_posixsubprocess.c

Comment thread Modules/_posixsubprocess.c

gpshead reviewed Jan 26, 2019

View reviewed changes

Comment thread Modules/_posixsubprocess.c

Comment thread Modules/_posixsubprocess.c

izbyshev commented Jan 30, 2019

View reviewed changes

Comment thread Modules/_posixsubprocess.c Outdated

izbyshev force-pushed the vfork branch from 789bbcc to c8353a0 Compare October 16, 2020 05:07

izbyshev changed the title bpo-35823: subprocess: Use vfork() instead of fork() on Linux bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe Oct 16, 2020

izbyshev force-pushed the vfork branch from c8353a0 to c81bf6e Compare October 22, 2020 02:11

izbyshev commented Oct 22, 2020

View reviewed changes

Comment thread Modules/_posixsubprocess.c

gpshead self-assigned this Oct 22, 2020

Prevent vfork() when call_setsid is set.

f226d33

It isn't clear if setsid() is safe after vfork(); better safe than sorry. Also adds a couple comments and some debug build assertions.

gpshead approved these changes Oct 24, 2020

View reviewed changes

Comment thread Modules/_posixsubprocess.c

Comment thread Modules/_posixsubprocess.c

Comment thread Modules/_posixsubprocess.c Outdated

bedevere-bot added awaiting merge and removed awaiting review labels Oct 24, 2020

gpshead added sprint type-feature A feature request or enhancement performance Performance or resource usage labels Oct 24, 2020

gpshead merged commit 976da90 into python:master Oct 24, 2020

bedevere-bot removed the awaiting merge label Oct 24, 2020

izbyshev deleted the vfork branch October 24, 2020 14:51

bdraco mentioned this pull request Feb 12, 2023

Avoid subprocess memory copy when c library supports posix_spawn home-assistant/core#87958

Merged

19 tasks

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe by izbyshev · Pull Request #11671 · python/cpython · GitHub

Uh oh!

Uh oh!

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe - #11671

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe#11671
gpshead merged 2 commits into
python:masterfrom
izbyshev:vfork

izbyshev commented Jan 25, 2019 •
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gpshead left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

izbyshev commented Oct 22, 2020

Uh oh!

gpshead left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Oct 24, 2020

Uh oh!

bedevere-bot commented Oct 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe by izbyshev · Pull Request #11671 · python/cpython · GitHub

Uh oh!

Uh oh!

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe - #11671

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe#11671gpshead merged 2 commits intopython:masterpython/cpython:masterfrom izbyshev:vforkizbyshev/cpython:vforkCopy head branch name to clipboard

Conversation

izbyshev commented Jan 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gpshead left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

izbyshev commented Oct 22, 2020

Uh oh!

gpshead left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-bot commented Oct 24, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Oct 24, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe#11671
gpshead merged 2 commits into
python:masterfrom
izbyshev:vfork

izbyshev commented Jan 25, 2019 •
edited
Loading