FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-124218: Use per-thread refcounts for code objects by colesbury · Pull Request #125216 · python/cpython · GitHub

/ cpython Public

gh-124218: Use per-thread refcounts for code objects - #125216

Merged
colesbury merged 3 commits into
python:mainfrom
colesbury:gh-124218-thread-refcount
Oct 15, 2024
ConversationCommits (3)ChecksFiles changed
Merged

gh-124218: Use per-thread refcounts for code objects#125216
colesbury merged 3 commits into
python:mainfrom
colesbury:gh-124218-thread-refcount

Conversation

colesbury commented Oct 9, 2024 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Use per-thread refcounting for the reference from function objects to their corresponding code object. This can be a source of contention when frequently creating nested functions. Deferred refcounting alone isn't a great fit here because these references are on the heap and may be modified by other libraries.

We will still need to address the globals and builtins dictionaries in a similar way.

pythongh-124218: Use per-thread refcounts for code objects
018b2a1 
Use per-thread refcounting for the reference from function objects to
their corresponding code object. This can be a source of contention when
frequently creating nested functions. Deferred refcounting alone isn't a
great fit here because these references are on the heap and may be
modified by other libraries.
bedevere-app Bot mentioned this pull request Oct 9, 2024
Closed
colesbury added skip news 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section labels Oct 9, 2024

bedevere-bot commented Oct 9, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 018b2a1 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Oct 9, 2024
colesbury marked this pull request as ready for review October 10, 2024 13:32
colesbury requested a review from DinoV October 10, 2024 13:32

markshannon commented Oct 10, 2024

Copy link
Copy Markdown
Member

This seems like we are adding more code, rather than reusing the existing code for per-thread refcounts already in existence for classes.
Can we factor out this code?

colesbury commented Oct 10, 2024

Copy link
Copy Markdown
Contributor Author

It is factored out and reusing existing code. That's why _Py_INCREF_CODE and _Py_DECREF_CODE are only one line each.

What concretely would you like changed?

markshannon commented Oct 10, 2024

Copy link
Copy Markdown
Member

Quite a lot of code is being added to pycore_object.h, but I don't see code being removed from typeobject.c or wherever the code for per-thread reference counts for classes is.
I would have though that classes could use the new code as well.

colesbury commented Oct 10, 2024

Copy link
Copy Markdown
Contributor Author

What would you like to see changed? pycore_object.h is +22 lines. That doesn't seem like quite a lot to me. That is:

  • Add two simple functions with one line bodies (_Py_INCREF_CODE and _Py_DECREF_CODE): +12
  • Split off the generic _Py_THREAD_INCREF_OBJECT from _Py_INCREF_TYPE: +5
  • Split off the generic _Py_THREAD_DECREF_OBJECT from _Py_DECREF_TYPE: +5

colesbury requested a review from mpage October 15, 2024 15:25
kumaraditya303 reviewed Oct 15, 2024
View reviewed changes
Comment thread Python/uniqueid.c Outdated
kumaraditya303 reviewed Oct 15, 2024
View reviewed changes
Comment thread Include/internal/pycore_object.h
static inline void
_Py_DECREF_CODE(PyCodeObject *co)
{
_Py_THREAD_DECREF_OBJECT((PyObject *)co, co->_co_unique_id);

kumaraditya303 Oct 15, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

assert that co is a code object.

colesbury Oct 15, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

co is statically a PyCodeObject *. We don't typically dynamically check the type when we have the static type (e.g., in PyCode_Addr2Line)

kumaraditya303 Oct 15, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I meant it to be protect against incorrect casts, anyways it's minor.

kumaraditya303 reviewed Oct 15, 2024
View reviewed changes
Comment thread Include/internal/pycore_object.h
static inline void
_Py_INCREF_CODE(PyCodeObject *co)
{
_Py_THREAD_INCREF_OBJECT((PyObject *)co, co->_co_unique_id);

kumaraditya303 Oct 15, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Same add assert to check it is codeobject

Update Python/uniqueid.c
7ad129b 
Co-authored-by: Kumar Aditya <kumaraditya@python.org>

colesbury commented Oct 15, 2024

Copy link
Copy Markdown
Contributor Author

!buildbot AMD64 Fedora Stable Refleaks

bedevere-bot commented Oct 15, 2024

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 7ad129b 🤖

The command will test the builders whose names match following regular expression: AMD64 Fedora Stable Refleaks

The builders matched are:

  • AMD64 Fedora Stable Refleaks PR

mpage approved these changes Oct 15, 2024
View reviewed changes

mpage left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM - just some comment fixups inline.

Comment thread Include/internal/pycore_object.h Outdated
Comment thread Include/internal/pycore_object.h Outdated
Comment thread Python/uniqueid.c Outdated
Apply suggestions from code review
89fa5db 
Co-authored-by: mpage <mpage@cs.stanford.edu>
colesbury merged commit 3ea488a into python:main Oct 15, 2024
colesbury deleted the gh-124218-thread-refcount branch October 15, 2024 19:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

mpage mpage approved these changes
kumaraditya303 kumaraditya303 approved these changes
ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner
markshannon Awaiting requested review from markshannon markshannon is a code owner
DinoV Awaiting requested review from DinoV

Assignees

No one assigned

Labels

skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

Back | FazBrowse Home | New Git URL