FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-126316: Make grp.getgrall() thread-safe: add a mutex by vstinner · Pull Request #127055 · python/cpython · GitHub

Repository navigation

gh-126316: Make grp.getgrall() thread-safe: add a mutex - #127055

Merged
vstinner merged 4 commits into
python:mainfrom
vstinner:getgrall_mutex
Nov 21, 2024
Merged

vstinner merged 4 commits into
python:mainfrom
vstinner:getgrall_mutex

Conversation

vstinner commented Nov 20, 2024 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

grpmodule.c is no longer built with the limited C API, since PyMutex is excluded from the limited C API.

grpmodule.c is no longer built with the limited C API, since PyMutex
is excluded from the limited C API.

Copy link
Copy Markdown
Member Author

@ZeroIntensity @colesbury: Would you mind to review my change?

ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hmm, this is an interesting way to do it. Local static variables are notorious for being non-thread-safe, so I'm not sure this kind of approach will work. If it does, I don't think we're fixing any issues with re-entrancy.

I would just go with @critical_section on each function, and then switch the module's Py_mod_multiple_interpreters to Py_MOD_MULTIPLE_INTERPRETERS_SUPPORTED instead of Py_MOD_PER_INTERPRETER_GIL_SUPPORTED to denote shared-GIL for subinterpreters. That should fix races for both cases.

colesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This approach looks good to me. I think you are right that we don't have to worry about deadlock here because the calls inside the lock won't call arbitrary Python code.

There's a failing CI related to generated files. I think the pycore_lock.h include in pycore_modsupport.h can be removed now. Also, maybe the arg clinic files need to be regenerated?

Local static variables are notorious for being non-thread-safe

A static PyMutex is fine because PyMutex is thread-safe. (It doesn't matter if it's a local or global static variable.)

The general problems with static variables (including static local variables) is that the data structures are not thread-safe or that they are lazily initialized pointers, which is not the case here.

Copy link
Copy Markdown
Member

Thanks for the insight, Sam. How scalable is this approach (as in, will we have a static PyMutex everywhere)?

Copy link
Copy Markdown
Contributor

I don't think we'll have static PyMutex everywhere, but this is a good pattern for certain libc functions that are not thread-safe and do not have thread-safe alternatives.

Copy link
Copy Markdown
Member Author

I think the pycore_lock.h include in pycore_modsupport.h can be removed now.

Correct: I wrote #127093 to remove it.

Comment thread Modules/grpmodule.c

ZeroIntensity left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

vstinner merged commit 3c2bd66 into python:main Nov 21, 2024
vstinner deleted the getgrall_mutex branch November 21, 2024 14:47

Copy link
Copy Markdown
Member Author

Merged, thanks for reviews @colesbury and @ZeroIntensity.

bedevere-app Bot commented Nov 21, 2024

Copy link
Copy Markdown

GH-127104 is a backport of this pull request to the 3.13 branch.

vstinner added a commit to vstinner/cpython that referenced this pull request Nov 21, 2024
…#127055)

grpmodule.c is no longer built with the limited C API, since PyMutex
is excluded from the limited C API.

(cherry picked from commit 3c2bd66)
vstinner added a commit that referenced this pull request Nov 26, 2024
…) (#127104)

* gh-126316: Make grp.getgrall() thread-safe: add a mutex (#127055)

grpmodule.c is no longer built with the limited C API, since PyMutex
is excluded from the limited C API.

(cherry picked from commit 3c2bd66)

* Revert ABI changes

Don't use Argument Clinic for grp.getgrgid() to avoid changing the
ABI (change PyInterpreterState structure by adding an "id"
identifier).
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…#127055)

grpmodule.c is no longer built with the limited C API, since PyMutex
is excluded from the limited C API.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL