gh-139103: Improve namedtuple scaling in free-threaded build - #144332
Conversation
Sorry, something went wrong.
Add `_Py_type_getattro_stackref`, a variant of type attribute lookup that returns `_PyStackRef` instead of `PyObject*`. This allows returning deferred references in the free-threaded build, reducing reference count contention when accessing type attributes. This significantly improves scaling of namedtuple instantiation across multiple threads.
|
FYI, I think I'm going to try splitting out parts of this into other PRs. |
Sorry, something went wrong.
Thanks, since this PR is quite big :-) |
Sorry, something went wrong.
| staticmethod *sm = (staticmethod *) | ||
| PyType_GenericAlloc(&PyStaticMethod_Type, 0); | ||
| if (sm != NULL) { | ||
| _PyObject_SetDeferredRefcount((PyObject *)sm); |
There was a problem hiding this comment.
Should we also call it in sm_init()? If not, should we move this _PyObject_SetDeferredRefcount() call to typeobject.c after the two PyStaticMethod_New() calls?
Sorry, something went wrong.
There was a problem hiding this comment.
Hmmm... maybe? We'll need to guard the call to _PyObject_SetDeferredRefcount() because calling it multiple times on the same object will trigger an assertion failure.
I'll update this after your PR is merged
Sorry, something went wrong.
There was a problem hiding this comment.
I merged for classmethod/staticmethod fix in the main branch.
Sorry, something went wrong.
There was a problem hiding this comment.
I've put the calls to _PyObject_SetDeferredRefcount() in sm_new() and PyStaticMethod_New() so that it's called exactly once during construction.
Sorry, something went wrong.
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner
left a comment
There was a problem hiding this comment.
LGTM. The C changes LGTM.
Sorry, something went wrong.
Co-authored-by: Victor Stinner <vstinner@python.org>
…ythongh-144332) Add `_Py_type_getattro_stackref`, a variant of type attribute lookup that returns `_PyStackRef` instead of `PyObject*`. This allows returning deferred references in the free-threaded build, reducing reference count contention when accessing type attributes. This significantly improves scaling of namedtuple instantiation across multiple threads. * Add blurb * Rename PyObject_GetAttrStackRef to _PyObject_GetAttrStackRef * Apply suggestion from @vstinner Co-authored-by: Victor Stinner <vstinner@python.org> * Apply suggestion from @vstinner Co-authored-by: Victor Stinner <vstinner@python.org> * format * Update Include/internal/pycore_function.h Co-authored-by: Victor Stinner <vstinner@python.org> --------- Co-authored-by: Victor Stinner <vstinner@python.org>
…ythongh-144332) Add `_Py_type_getattro_stackref`, a variant of type attribute lookup that returns `_PyStackRef` instead of `PyObject*`. This allows returning deferred references in the free-threaded build, reducing reference count contention when accessing type attributes. This significantly improves scaling of namedtuple instantiation across multiple threads. * Add blurb * Rename PyObject_GetAttrStackRef to _PyObject_GetAttrStackRef * Apply suggestion from @vstinner Co-authored-by: Victor Stinner <vstinner@python.org> * Apply suggestion from @vstinner Co-authored-by: Victor Stinner <vstinner@python.org> * format * Update Include/internal/pycore_function.h Co-authored-by: Victor Stinner <vstinner@python.org> --------- Co-authored-by: Victor Stinner <vstinner@python.org>
Add _Py_type_getattro_stackref, a variant of type attribute lookup that returns _PyStackRef instead of PyObject*. This allows returning deferred references in the free-threaded build, reducing reference count contention when accessing type attributes.
This improves scaling of namedtuple instantiation across multiple threads.