FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-39877: take_gil() now exits the thread if finalizing by vstinner · Pull Request #18854 · python/cpython · GitHub

/ cpython Public

bpo-39877: take_gil() now exits the thread if finalizing - #18854

Closed
vstinner wants to merge 2 commits into
python:masterfrom
vstinner:take_gil_exit
ConversationCommits (2)ChecksFiles changed
Closed

bpo-39877: take_gil() now exits the thread if finalizing#18854
vstinner wants to merge 2 commits into
python:masterfrom
vstinner:take_gil_exit

Conversation

vstinner commented Mar 8, 2020 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

take_gil() is now responsible to exit the thread if Python is
finalizing. Not only exit before trying to acquire the GIL, but exit
also once the GIL is succesfully acquired.

https://bugs.python.org/issue39877

bpo-39877: take_gil() now exits the thread if finalizing
9ac0b25 
take_gil() is now responsible to exit the thread if Python is
finalizing. Not only exit before trying to acquire the GIL, but exit
also once the GIL is succesfully acquired.

vstinner commented Mar 8, 2020

Copy link
Copy Markdown
Member Author

@pitrou: My previous change introduced a race condition: https://bugs.python.org/issue39877#msg363667 This change should fix it.

vstinner commented Mar 8, 2020 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member Author

I tested manually: with this change and PR #18848, asyncio_gc.py of bpo-19466 does no longer crash.

pitrou requested changes Mar 8, 2020
View reviewed changes

pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I believe the fix is somehow incomplete, see below.
Also, as you say, the problem is "is it safe to destroy the GIL if there are still daemon threads trying to take it?".

Comment thread Python/ceval_gil.h Outdated

MUTEX_UNLOCK(gil->mutex);

if (thread_must_exit(tstate)) {

pitrou Mar 8, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure why you're doing this here. It doesn't seem necessary and, furthermore, it's unlikely to trigger.

vstinner Mar 8, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This code path is to prevent https://bugs.python.org/issue39877#msg363667 crash. Are you suggesting to move this code at line 249, after COND_TIMED_WAIT()?

thread_must_exit() at entry is to prevent https://bugs.python.org/issue39877#msg363512 crash.

Comment thread Python/ceval_gil.h
{
int err = errno;

if (thread_must_exit(tstate)) {

pitrou Mar 8, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Ok, but shouldn't you do this after COND_TIMED_WAIT below as well?

bedevere-bot commented Mar 8, 2020

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

vstinner commented Mar 8, 2020
View reviewed changes
Comment thread Python/ceval_gil.h
to run after wait_for_thread_shutdown() and before Py_Finalize()
completes. For example, when _PyImport_Cleanup() executes Python
code. */
MUTEX_UNLOCK(gil->mutex);

vstinner Mar 8, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure if anyone else should be done here.

vstinner Mar 9, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

For example, is COND_SIGNAL(gil->cond) needed to wake up the next thread waiting on COND_TIMED_WAIT()?

vstinner commented Mar 9, 2020

Copy link
Copy Markdown
Member Author

I have made the requested changes; please review again.

bedevere-bot commented Mar 9, 2020

Copy link
Copy Markdown

Thanks for making the requested changes!

@pitrou: please review the changes made to this pull request.

bedevere-bot requested a review from pitrou March 9, 2020 07:44
vstinner mentioned this pull request Mar 9, 2020
Merged

vstinner commented Mar 9, 2020

Copy link
Copy Markdown
Member Author

I merged PR #18890 which checks if the thread must exit at function exit as well. In short, it restores the Python 3.8 behavior.

I close this PR for now.

We can revisit take_gil() later to try to adjust/optimize it later. I prefer to move step by step.

vstinner closed this Mar 9, 2020
vstinner deleted the take_gil_exit branch March 9, 2020 22:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

pitrou Awaiting requested review from pitrou

Assignees

No one assigned

Labels

awaiting change review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

Back | FazBrowse Home | New Git URL