FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Add _DuringGC functions for tp_traverse · Issue #145921 · python/cpython · GitHub

Repository navigation

Add _DuringGC functions for tp_traverse #145921

Open

Description

Feature or enhancement

As seen in https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/13 and capi-workgroup/decisions#95:

There are newly documented restrictions on tp_traverse:

The traversal function must not have any side effects. It must not modify the reference counts of any Python objects nor create or destroy any Python objects.

But, whether a C-API function modifies some reference count is an implementation detail that we can change at any time. Core devs can add logging, an audit event, or a warning, to virtually any function.
This implies that you can't safely use any C API function in tp_traverse, unless it guarantees that it has no side effects.

I propose to add such side-effect-free versions of some functions useful in tp_traverse, with a _DuringGC suffix -- namely:

  • PyObject_GetTypeData_DuringGC
  • PyObject_GetItemData_DuringGC
  • PyType_GetModuleState_DuringGC
  • PyModule_GetState_DuringGC
  • PyModule_GetToken_DuringGC
  • PyType_GetBaseByToken_DuringGC
  • PyType_GetModule_DuringGC
  • PyType_GetModuleByToken_DuringGC

Compared to their non-suffixed variants, the main difference is that we promise that these will remain safe to use in tp_traverse. The other differences are that these functions cannot fail, and (where applicable) they return borrowed references rather than strong ones.

Also, Py_VISIT, Py_SIZE, PyObject_VisitManagedDict as well as the visitproc that Python passes to tp_traverse will be documented to work like this.
[edit] I'll also include a few ones useful in these: PyObject_TypeCheck, PyType_IsSubtype, PyType_HasFeature, Py{<type>}_Check, Py{<type>}_CheckExact.

Users that call any _DuringGC function (including some type's tp_traverse) must manually must ensure that the type, MRO, and layout of any argument doesn't concurrently change. That means only calling them from tp_traverse.

Linked PRs

Activity

  1. changed the title [-]Add functions tp_traverse[/-] [+]Add _DuringGC functions for tp_traverse[/+] on Mar 13, 2026
  2. added a commit that references this issue on Mar 13, 2026
  3. deleted a comment from khalidsaidi on Mar 14, 2026
  4. encukou commented on Apr 8, 2026

    MemberAuthor

    I've merged with a few outstanding issues in tp_traverse documentation that Steve pointed out. Both are for pre-existing functionality.

    On the note “The tp_traverse function can be called from any thread.”:

    Aren't these new functions explicitly not thread safe, though? If we can't guarantee that you can safely access your objects during tp_traverse, what are we expecting - authors should figure out their own way to lock all other Python threads? How are they meant to coordinate it?

    Maybe there's a way to describe it that says "the entire object graph is guaranteed to not be changed by any other Python thread" that leaves open the possibility of those threads not being blocked, but also not being able to interfere?

    And on the traverse function's result:

    If they fail during tp_traverse, should we be bailing out as quickly as possible? What value should we return?

    →

    would it be correct to say tp_traverse should return zero on recoverable errors, including being unable to visit some objects?
    Currently gc.c ignores errors*, but non-zero a result makes VISIT return early; gc_free_threading.c calls PyErr_NoMemory if traversal fails.

  5. added a commit that references this issue on Apr 8, 2026
  6. added a commit that references this issue on Apr 25, 2026
  7. encukou commented on Oct 22, 2026

    MemberAuthor

    @nascheme, maybe you could help with the docs/recommendations here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions


      Back | FazBrowse Home | New Git URL