gh-153400: Use glibc functions instead of syscall() by vstinner · Pull Request #155518 · python/cpython · GitHub

Use glibc functions instead of syscall(): pidfd_open(), pidfd_getfd() and pidfd_send_signal() (glibc 2.36), gettid() and getdents64() (glibc 2.30), and getrandom() (glibc 2.25).

Use unsigned int for os.getrandom() flags and signal.pidfd_send_signal() flags.

Issue: Resolve newer Linux os functions at runtime instead of compiling them out against old glibc #153400

Use glibc functions instead of syscall(): pidfd_open(), pidfd_getfd() and pidfd_send_signal() (glibc 2.36), gettid() and getdents64() (glibc 2.30), and getrandom() (glibc 2.25).

Use unsigned int for os.getrandom() flags and signal.pidfd_send_signal() flags.

@thesamesam:

For some syscall wrappers, glibc has _FORTIFY_SOURCE checks on the arguments. CPython right now doesn't ever use glibc wrappers (uses syscall instead), it just uses glibc for the syscall numbers, which means the status quo is the worst-of-both-worlds: you get no protection or sanity checks, but you also need a newer glibc that acknowledges the syscall exists. The number could also come from linux-headers.

I think CPython should use the glibc wrappers if they exist at build-time at least.

I wrote this change to replace syscall() calls with glibc function calls here instead. Is it what you proposed?

🤖 New build scheduled with the buildbot fleet by @vstinner for commit b6bbfd9 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F155518%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

I wrote this change to replace syscall() calls with glibc function calls here instead. Is it what you proposed?

It looks good, thank you!

buildbot/AMD64 Android PR — Build done.

The "test" step failed with a timeout, not good. I'm not sure if it's related to the change or not.

!buildbot AMD64 Android

🤖 New build scheduled with the buildbot fleet by @mhsmith for commit b6bbfd9 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F155518%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 Android

The builders matched are:

AMD64 Android PR

The buildbot stopped working about 24 hours ago, so it's not related to this change. Testing manually, it appeared to work on the first run within each Gradle daemon, but if I interrupted it and tried again, it hung while starting the emulator. Removing the emulator snapshot and allowing it to be recreated seems to have fixed the issue.

No, apparently not, and it looks like this previously happened for a few runs in a row on August 5, then again on August 6.

In each case, it was triggered by a run that was cancelled for some reason, such as a master shutdown. Subsequent runs then stop at the line > Task :app:maxVersionDebugAndroidTest, and time out after 10 minutes without having started the emulator. The problem persists until there's a long enough gap between runs for the broken Gradle daemon to exit from inactivity. So this was probably caused by the Gradle update on August 1 (#155012).

I'll shut the buildbot down until I have a chance to look into this. Notes to self:

Check the version of platform-tools, emulator, or any other SDK packages that may be involved. Match the versions on GitHub Actions, or the other buildbot, but keep a record of the previous versions (see notes in Android buildbot failures after updating to API level 35 #142387).
Switch to Java 25, since that's what this version of AGP is best tested with.
Switch to an older version of AGP which is still compatible with Java 25.
Revert the update completely.

We still have x86_64 being tested by GitHub Actions, and aarch64 by the other buildbot, so it doesn't need to block this PR.

) Use glibc functions instead of syscall(): pidfd_open(), pidfd_getfd() and pidfd_send_signal() (glibc 2.36), gettid() and getdents64() (glibc 2.30), and getrandom() (glibc 2.25). Use unsigned int for os.getrandom() flags and signal.pidfd_send_signal() flags.

I didn't try any of the ideas above, because although I reproduced the problem manually several times, I wasn't able to do it reliably, so I'd have no way of knowing whether anything worked. None of the following things were good enough:

Interrupting multiple runs in a row
Interrupting quickly or slowly
Running git clean or gradlew clean between runs

Instead, since the problem involves the Gradle daemon getting into a broken state, I disabled persistent daemons by adding this line to the worker startup script:

export GRADLE_OPTS="-Dorg.gradle.daemon=false"

This should make each run more independent, as it is on GitHub Actions. It only costs about 20 seconds per run.

LGTM, thanks!

[post merge review] confirming that this is safe. glibc's struct dirent64 is as above and has 256 bytes of d_name space declared. I think the linux kernel itself uses a sizeless d_name[] to expect the caller to allocate extra. if that happened here we'd wind up with a tiny struct without necessarily room for a name and an error from the syscall, silently falling back to a slower or unsafe code path instead.

I disabled persistent daemons by adding this line to the worker startup script:
export GRADLE_OPTS="-Dorg.gradle.daemon=false"

The same problem has now also occurred on the aarch64 buildbot, once on 09-13 and once on 09-15. So I've moved the workaround to the buildmaster config (python/buildmaster-config#796).

Since it's now happened on both machines, that makes it much less likely to be caused by the version of Java or any SDK packages, and more likely to be caused by the version of Gradle or the AGP.

Identical symptoms appeared again today on the aarch64 buildbot, after losing the connection to the master. However, it did not recover even after a 3 hour gap, and the daemon was disabled anyway. Removing the emulator snapshot fixed the issue.

If you have more issues with Android, you may open a separated issue, since this issue is closed and not directly related to Android.

vstinner requested review from AA-Turner, corona10, emmatyping, erlend-aasland and itamaro as code owners August 10, 2026 20:12

bedevere-app Bot mentioned this pull request Aug 10, 2026

Resolve newer Linux os functions at runtime instead of compiling them out against old glibc #153400

Closed

bedevere-app Bot added the awaiting core review label Aug 10, 2026

vstinner mentioned this pull request Aug 10, 2026

gh-153400: Resolve newer libc/syscall wrappers at runtime on Linux #154816

Closed

pythongh-153400 : Use glibc functions instead of syscall()

c99f647

Use glibc functions instead of syscall(): pidfd_open(), pidfd_getfd() and pidfd_send_signal() (glibc 2.36), gettid() and getdents64() (glibc 2.30), and getrandom() (glibc 2.25).

vstinner force-pushed the pidfd branch from 8c0011d to c99f647 Compare August 10, 2026 21:06

vstinner requested a review from gpshead as a code owner August 10, 2026 21:06

vstinner changed the title gh-153400: Use pidfd_open() and pidfd_getfd() functions gh-153400: Use glibc functions instead of syscall() Aug 10, 2026

Use unsigned int for os.getrandom() flags

b6bbfd9

Use unsigned int for os.getrandom() flags and signal.pidfd_send_signal() flags.

vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 11, 2026

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 11, 2026

vstinner merged commit 1c9521f into python:main Aug 13, 2026
141 of 142 checks passed

vstinner deleted the pidfd branch August 13, 2026 10:44

bedevere-app Bot removed the awaiting core review label Aug 13, 2026

vstinner mentioned this pull request Aug 13, 2026

gh-153400: Add syscall fallbacks for copy_file_range/memfd_create #155520

Merged

picnixz mentioned this pull request Aug 14, 2026

gh-153400: Support os.getrandom() on old glibc versions #155762

Merged

gpshead reviewed Sep 7, 2026

View reviewed changes

mhsmith mentioned this pull request Sep 16, 2026

In Android tests, disable Gradle daemon python/buildmaster-config#796

Merged

-                                              (struct linux_dirent64 *)buffer,
-                                              sizeof(buffer))) > 0) {
-                          struct linux_dirent64 *entry;
+                      char buffer[sizeof(struct py_dirent64)];

gh-153400: Use glibc functions instead of syscall() by vstinner · Pull Request #155518 · python/cpython · GitHub

Repository navigation

gh-153400: Use glibc functions instead of syscall() - #155518

gh-153400: Use glibc functions instead of syscall()#155518vstinner merged 2 commits intopython:mainpython/cpython:mainfrom vstinner:pidfdvstinner/cpython:pidfdCopy head branch name to clipboard

Conversation

vstinner commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Aug 11, 2026

Uh oh!

bedevere-bot commented Aug 11, 2026

Uh oh!

thesamesam commented Aug 11, 2026

Uh oh!

vstinner commented Aug 11, 2026

Uh oh!

mhsmith commented Aug 11, 2026

Uh oh!

bedevere-bot commented Aug 11, 2026

Uh oh!

mhsmith commented Aug 11, 2026

Uh oh!

mhsmith commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mhsmith commented Sep 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpshead left a comment

Choose a reason for hiding this comment

Uh oh!

gpshead Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

mhsmith commented Sep 16, 2026

Uh oh!

mhsmith commented Sep 21, 2026

Uh oh!

vstinner commented Sep 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-153400: Use glibc functions instead of syscall()#155518
vstinner merged 2 commits into
python:mainfrom
vstinner:pidfd

vstinner commented Aug 10, 2026 •
edited
Loading

mhsmith commented Aug 11, 2026 •
edited
Loading

mhsmith commented Sep 7, 2026 •
edited
Loading