FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-133467: Fix typeobject `tp_base` race in free threading by LindaSummer · Pull Request #140549 · python/cpython · GitHub

/ cpython Public

gh-133467: Fix typeobject tp_base race in free threading - #140549

Merged
colesbury merged 6 commits into
python:mainfrom
LindaSummer:bugfix/type-race
Nov 5, 2025
ConversationCommits (6)ChecksFiles changed
Merged

gh-133467: Fix typeobject tp_base race in free threading#140549
colesbury merged 6 commits into
python:mainfrom
LindaSummer:bugfix/type-race

Conversation

LindaSummer commented Oct 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Issue

#133467

Proposed Changes

  • Fix the race of PyTypeObject::tp_base between typemember __base__ and set of __bases__.
  • Add test case in unit test.

Comment

Root cause

Since the __bases__'s getter and setter are protetcted by TYPE_LOCK.

cpython/Objects/typeobject.c

Line 1916 in 289360a

ASSERT_TYPE_LOCK_HELD();

cpython/Objects/typeobject.c

Lines 586 to 596 in 289360a

_PyType_GetBases(PyTypeObject *self)
{
PyObject *res;
BEGIN_TYPE_LOCK();
res = lookup_tp_bases(self);
Py_INCREF(res);
END_TYPE_LOCK();
return res;
}

So it is safe for concurrency.

But the update of bases will re-evaluate the tp_base again and would be in race with the direct access of __base__.

cpython/Objects/typeobject.c

Line 1459 in 289360a

{"__base__", _Py_T_OBJECT, offsetof(PyTypeObject, tp_base), Py_READONLY},

How to fix

At the beginning I try to use the TYPE_LOCK to protect the tb_base for __base__ access.

But it's a directly memory access by offset and I have no more idea except StopTheWorld.

So I add a STW for the updating of tp_base in setter.

Please let me know if we have a better way for it. 😊

TSAN output for this race

Here is the TSAN core output of this race.

WARNING: ThreadSanitizer: data race (pid=1754505)
  Write of size 8 at 0x7fb1ed269910 by thread T27:
    #0 type_set_bases_unlocked /home/someuser/project/cpython/Objects/typeobject.c:1924:19 (python+0x6b265a) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #1 type_set_bases /home/someuser/project/cpython/Objects/typeobject.c:2009:15 (python+0x6b265a)
    #2 getset_set /home/someuser/project/cpython/Objects/descrobject.c:250:16 (python+0x5a8fe6) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #3 type_setattro /home/someuser/project/cpython/Objects/typeobject.c:6532:19 (python+0x6a2ac5) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #4 PyObject_SetAttr /home/someuser/project/cpython/Objects/object.c:1476:15 (python+0x649360) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #5 _PyEval_EvalFrameDefault /home/someuser/project/cpython/Python/generated_cases.c.h:10750:27 (python+0x7bd953) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #6 _PyEval_EvalFrame /home/someuser/project/cpython/./Include/internal/pycore_ceval.h:121:16 (python+0x79c3a0) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #7 _PyEval_Vector /home/someuser/project/cpython/Python/ceval.c:2005:12 (python+0x79c3a0)
    #8 _PyFunction_Vectorcall /home/someuser/project/cpython/Objects/call.c (python+0x590def) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #9 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x5956b6) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #10 method_vectorcall /home/someuser/project/cpython/Objects/classobject.c:73:20 (python+0x5956b6)
    #11 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x7f4a21) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #12 context_run /home/someuser/project/cpython/Python/context.c:728:29 (python+0x7f4a21)
    #13 _PyEval_EvalFrameDefault /home/someuser/project/cpython/Python/generated_cases.c.h:3710:35 (python+0x7a9401) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #14 _PyEval_EvalFrame /home/someuser/project/cpython/./Include/internal/pycore_ceval.h:121:16 (python+0x79c3a0) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #15 _PyEval_Vector /home/someuser/project/cpython/Python/ceval.c:2005:12 (python+0x79c3a0)
    #16 _PyFunction_Vectorcall /home/someuser/project/cpython/Objects/call.c (python+0x590def) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #17 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x5956b6) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #18 method_vectorcall /home/someuser/project/cpython/Objects/classobject.c:73:20 (python+0x5956b6)
    #19 _PyVectorcall_Call /home/someuser/project/cpython/Objects/call.c:273:16 (python+0x590a87) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #20 _PyObject_Call /home/someuser/project/cpython/Objects/call.c:348:16 (python+0x590a87)
    #21 PyObject_Call /home/someuser/project/cpython/Objects/call.c:373:12 (python+0x590af5) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #22 thread_run /home/someuser/project/cpython/./Modules/_threadmodule.c:387:21 (python+0x96c152) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #23 pythread_wrapper /home/someuser/project/cpython/Python/thread_pthread.h:234:5 (python+0x8a0317) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)

  Previous atomic read of size 8 at 0x7fb1ed269910 by thread T22:
    #0 _Py_atomic_load_ptr /home/someuser/project/cpython/./Include/cpython/pyatomic_gcc.h:300:18 (python+0x886ef1) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #1 _Py_TryIncrefCompare /home/someuser/project/cpython/./Include/internal/pycore_object.h:603:15 (python+0x886ef1)
    #2 PyMember_GetOne /home/someuser/project/cpython/Python/structmember.c:91:18 (python+0x8869ca) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #3 member_get /home/someuser/project/cpython/Objects/descrobject.c:180:12 (python+0x5a8a06) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #4 _Py_type_getattro_impl /home/someuser/project/cpython/Objects/typeobject.c:6363:19 (python+0x6a0992) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #5 _Py_type_getattro /home/someuser/project/cpython/Objects/typeobject.c:6424:12 (python+0x6a0ca0) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #6 PyObject_GetAttr /home/someuser/project/cpython/Objects/object.c:1313:18 (python+0x648a3e) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #7 _PyObject_GetMethodStackRef /home/someuser/project/cpython/Objects/object.c:1706:25 (python+0x64b746) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #8 _PyEval_EvalFrameDefault /home/someuser/project/cpython/Python/generated_cases.c.h:7844:35 (python+0x7b55b4) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #9 _PyEval_EvalFrame /home/someuser/project/cpython/./Include/internal/pycore_ceval.h:121:16 (python+0x79c3a0) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #10 _PyEval_Vector /home/someuser/project/cpython/Python/ceval.c:2005:12 (python+0x79c3a0)
    #11 _PyFunction_Vectorcall /home/someuser/project/cpython/Objects/call.c (python+0x590def) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #12 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x5956b6) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #13 method_vectorcall /home/someuser/project/cpython/Objects/classobject.c:73:20 (python+0x5956b6)
    #14 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x7f4a21) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #15 context_run /home/someuser/project/cpython/Python/context.c:728:29 (python+0x7f4a21)
    #16 _PyEval_EvalFrameDefault /home/someuser/project/cpython/Python/generated_cases.c.h:3710:35 (python+0x7a9401) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #17 _PyEval_EvalFrame /home/someuser/project/cpython/./Include/internal/pycore_ceval.h:121:16 (python+0x79c3a0) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #18 _PyEval_Vector /home/someuser/project/cpython/Python/ceval.c:2005:12 (python+0x79c3a0)
    #19 _PyFunction_Vectorcall /home/someuser/project/cpython/Objects/call.c (python+0x590def) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #20 _PyObject_VectorcallTstate /home/someuser/project/cpython/./Include/internal/pycore_call.h:169:11 (python+0x5956b6) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #21 method_vectorcall /home/someuser/project/cpython/Objects/classobject.c:73:20 (python+0x5956b6)
    #22 _PyVectorcall_Call /home/someuser/project/cpython/Objects/call.c:273:16 (python+0x590a87) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #23 _PyObject_Call /home/someuser/project/cpython/Objects/call.c:348:16 (python+0x590a87)
    #24 PyObject_Call /home/someuser/project/cpython/Objects/call.c:373:12 (python+0x590af5) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #25 thread_run /home/someuser/project/cpython/./Modules/_threadmodule.c:387:21 (python+0x96c152) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)
    #26 pythread_wrapper /home/someuser/project/cpython/Python/thread_pthread.h:234:5 (python+0x8a0317) (BuildId: 111d1b338abaedccddf0af54b80c4dc6c30a747f)

bedevere-app Bot mentioned this pull request Oct 24, 2025
Closed
4 tasks
LindaSummer changed the title gh-133467: Fix tp_base race in free threading in typeobject gh-133467: Fix typeobject tp_base race in free threading Oct 24, 2025

LindaSummer commented Oct 24, 2025

Copy link
Copy Markdown
Contributor Author

Hi @nascheme,

Thansk very much for your guidance. 😊

I have created a case for the race of tb_base.

Please help take a look.

Best Regards,
Edward

LindaSummer force-pushed the bugfix/type-race branch 2 times, most recently from ba3206e to 0df5ffa Compare October 24, 2025 15:06
LindaSummer changed the title gh-133467: Fix typeobject tp_base race in free threading gh-133465: Fix typeobject tp_base race in free threading Oct 25, 2025
bedevere-app Bot mentioned this pull request Oct 25, 2025
Open
LindaSummer changed the title gh-133465: Fix typeobject tp_base race in free threading gh-133467: Fix typeobject tp_base race in free threading Oct 25, 2025

LindaSummer commented Oct 29, 2025

Copy link
Copy Markdown
Contributor Author

Hi @nascheme and @colesbury ,

Sorry to bother you. 😊
Could we take a review on this PR?

Hope to get suggestions from you!

Best Regards,
Edward

nascheme commented Oct 29, 2025

Copy link
Copy Markdown
Member

LGTM. Using types_stop_world() and types_start_world() would be a little cleaner, IMHO but that's minor.

colesbury commented Oct 29, 2025 via email

Copy link
Copy Markdown
Contributor

nascheme commented Oct 29, 2025

Copy link
Copy Markdown
Member

I think we might need the helper that avoids releasing the critical section
here.

We have this code in apply_type_slot_updates(), which I think could be used here too:

    type_lock_prevent_release();
    types_stop_world();
    apply_slot_updates(updates);
    types_start_world();
    type_lock_allow_release();

colesbury reviewed Oct 30, 2025
View reviewed changes
Comment thread Objects/typeobject.c
kumaraditya303 reviewed Nov 1, 2025
View reviewed changes
Comment thread Objects/typeobject.c Outdated
LindaSummer force-pushed the bugfix/type-race branch 3 times, most recently from f494440 to 9836fd7 Compare November 3, 2025 01:02

LindaSummer commented Nov 3, 2025

Copy link
Copy Markdown
Contributor Author

Hi @nascheme , @colesbury and @kumaraditya303 ,

I have updated the PR as suggested in the review comment. 😄

Please help take a review.

Wish you a good day!

Best Regards,
Edward

colesbury commented Nov 3, 2025

Copy link
Copy Markdown
Contributor

You'll need to update the calls to use type_lock_prevent_release() / type_lock_allow_release() because we do not want to release the type lock when stopping the world. See the pattern and comment in apply_type_slot_updates.

Also, please remove the associated suppression in Tools/tsan/suppressions_free_threading.txt

LindaSummer commented Nov 4, 2025

Copy link
Copy Markdown
Contributor Author

You'll need to update the calls to use type_lock_prevent_release() / type_lock_allow_release() because we do not want to release the type lock when stopping the world. See the pattern and comment in apply_type_slot_updates.

Also, please remove the associated suppression in Tools/tsan/suppressions_free_threading.txt

Hi @colesbury ,

Thanks very much for your help and suggestion! 😊

I have added the type_lock_prevent_release() / type_lock_allow_release() to guard the STW.

Please correct me if I misunderstand.

Best Regards,
Edward

kumaraditya303 reviewed Nov 4, 2025
View reviewed changes
Comment thread Objects/typeobject.c
LindaSummer requested a review from colesbury November 5, 2025 14:38

LindaSummer commented Nov 5, 2025

Copy link
Copy Markdown
Contributor Author

Hi @nascheme , @colesbury and @kumaraditya303 ,

Thanks very much for your patience and kind suggestions! 😊
I have updated PR as commented.
Please help take a review.

Wish you a good day!

Best Regards,
Edward

colesbury reviewed Nov 5, 2025
View reviewed changes
Comment thread Tools/tsan/suppressions_free_threading.txt Outdated
kumaraditya303 added the needs backport to 3.14 bugs and security fixes label Nov 5, 2025
colesbury approved these changes Nov 5, 2025
View reviewed changes

colesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM, thanks!

colesbury merged commit b83f379 into python:main Nov 5, 2025
52 checks passed

miss-islington-app Bot commented Nov 5, 2025

Copy link
Copy Markdown

Thanks @LindaSummer for the PR, and @colesbury for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington-app Bot commented Nov 5, 2025

Copy link
Copy Markdown

Sorry, @LindaSummer and @colesbury, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker b83f379a972c001864d3593cd64fc07e7c7f375f 3.14

colesbury commented Nov 5, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Backport probably depends on backporting:

nascheme commented Nov 7, 2025

Copy link
Copy Markdown
Member

Yes, this needs gh-133177 to be backported too and that's a pretty complicated change. So, it feels a bit risky to backport. My guess is that these races would be quite rare in real Python code. It's rare that Python code mutates types in ways that trigger these races. Maybe wait a bit more?

StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
pythongh-133467: Fix typeobject tp_base race in free threading (pyt…
59e12fb 
…hongh-140549)

serhiy-storchaka commented Dec 26, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Please don't forget about backporting or remove the "needs backport to 3.14" label if backporting is not planned.

kumaraditya303 removed the needs backport to 3.14 bugs and security fixes label Dec 26, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

colesbury colesbury approved these changes
kumaraditya303 kumaraditya303 approved these changes
markshannon Awaiting requested review from markshannon markshannon is a code owner
nascheme Awaiting requested review from nascheme

Assignees

colesbury

Labels

topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

Back | FazBrowse Home | New Git URL