FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

GH-99205: Mark new interpreters and threads as non-static by brandtbucher · Pull Request #99268 · python/cpython · GitHub

/ cpython Public

GH-99205: Mark new interpreters and threads as non-static - #99268

Merged
brandtbucher merged 4 commits into
python:mainfrom
brandtbucher:threadstate-memory-leak
Nov 9, 2022
ConversationCommits (4)ChecksFiles changed
Merged

GH-99205: Mark new interpreters and threads as non-static#99268
brandtbucher merged 4 commits into
python:mainfrom
brandtbucher:threadstate-memory-leak

Conversation

brandtbucher commented Nov 9, 2022 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

brandtbucher added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) needs backport to 3.11 only security fixes labels Nov 9, 2022
brandtbucher self-assigned this Nov 9, 2022
bedevere-bot mentioned this pull request Nov 9, 2022
Closed
kumaraditya303 approved these changes Nov 9, 2022
View reviewed changes

kumaraditya303 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

ericsnowcurrently approved these changes Nov 9, 2022
View reviewed changes

ericsnowcurrently left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

I've pointed out two places where a comment would be appropriate. I'm approving the PR under the assumption you'll take care of those.

Comment thread Python/pystate.c
Comment thread Python/pystate.c
memcpy(tstate,
&initial._main_interpreter._initial_thread,
sizeof(*tstate));
tstate->_static = false;

ericsnowcurrently Nov 9, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Please add a note pointing to PyThreadState_INIT() and indicating that fields set there should be adjusted here as appropriate.

Comment thread Python/pystate.c
// Set to _PyInterpreterState_INIT.
memcpy(interp, &initial._main_interpreter,
sizeof(*interp));
interp->_static = false;

ericsnowcurrently Nov 9, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Likewise, a note pointing to PyInterpreterState_INIT().

ericsnowcurrently commented Nov 9, 2022

Copy link
Copy Markdown
Member

Thanks for fixing this, BTW.

brandtbucher commented Nov 9, 2022

Copy link
Copy Markdown
Member Author

@pablogsal, leak tests didn't catch this because we use the "raw" domain to allocate PyThreadState and PyInterpreterState.

fuzzard mentioned this pull request Nov 9, 2022
Merged

pablogsal commented Nov 9, 2022

Copy link
Copy Markdown
Member

Oh interesting, can you confirm that changing the domain to memory makes the test fail?

brandtbucher commented Nov 9, 2022 •
edited
Loading

Copy link
Copy Markdown
Member Author

Oh interesting, can you confirm that changing the domain to memory makes the test fail?

You can't. :)

Fatal Python error: _PyMem_DebugCalloc: Python memory allocator called without holding the GIL
Python runtime state: preinitialized

See this comment:

cpython/Python/pystate.c

Lines 818 to 821 in 58ee5d8

// We don't need to allocate a thread state for the main interpreter
// (the common case), but doing it later for the other case revealed a
// reentrancy problem (deadlock). So for now we always allocate before
// taking the interpreters lock. See GH-96071.

brandtbucher merged commit 283ab0e into python:main Nov 9, 2022

miss-islington commented Nov 9, 2022

Copy link
Copy Markdown
Contributor

Thanks @brandtbucher for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

bedevere-bot commented Nov 9, 2022

Copy link
Copy Markdown

GH-99301 is a backport of this pull request to the 3.11 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 9, 2022
pythonGH-99205: Mark new interpreters and threads as non-static (pyth…
27763ef 
…onGH-99268)

(cherry picked from commit 283ab0e)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bedevere-bot removed the needs backport to 3.11 only security fixes label Nov 9, 2022
miss-islington added a commit that referenced this pull request Nov 9, 2022
GH-99205: Mark new interpreters and threads as non-static (GH-99268)
7c9c993 
(cherry picked from commit 283ab0e)

Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

ericsnowcurrently ericsnowcurrently approved these changes
kumaraditya303 kumaraditya303 approved these changes
pablogsal Awaiting requested review from pablogsal

Assignees

brandtbucher

Labels

interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

Back | FazBrowse Home | New Git URL