Repository navigation
gh-159564: Make weakref dereference lock-free. - #159578
Open
nascheme wants to merge 4 commits into
Open
nascheme wants to merge 4 commits into
nascheme wants to merge 4 commits into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
In the free-threaded build, _PyWeakref_GET_REF() no longer takes the striped weakref list lock. The referent's memory stays readable if it is freed concurrently because object heaps delay page reuse with QSBR, and wr_object only ever transitions from the referent to Py_None, so _Py_TryIncrefCompare() is sufficient. clear_weakref_lock_held() now uses release stores instead of sequentially consistent stores. The weakref list lock orders the list updates, and the lock-free read only requires that the store of Py_None happens before the referent's memory can be reused.
Comment thread
Include/internal/pycore_weakref.h
This block is GIL-enabled build only.
…free-weakref-get-ref
Even in the GIL-enabled build, the target's refcount can be zero here. Add a unit test that shows this and revert back to _Py_TryIncref.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In the free-threaded build, change _PyWeakref_GET_REF() so it no longer takes the striped weakref list lock. The referent's memory stays readable if it is freed concurrently because object heaps delay page reuse with QSBR, and wr_object only ever transitions from the referent to Py_None, so _Py_TryIncrefCompare() is sufficient.
Use release store for clear_weakref_lock_held(), instead of sequentially consistent stores. The weakref list lock orders the list updates, and the lock-free read only requires that the store of Py_None happens before the referent's memory can be reused.
The benchmark results below are based on a C extension implementing a weak-map using either PyUnstable_TryIncRef() (borrowed refereces) vs using weak references. Thread scaling of "get, N threads, private maps" (ns per op, lower is better):