gh-150411: fix `gc_generation.count` race by LindaSummer · Pull Request #150413 · python/cpython · GitHub

Issue

Root Cause

Refer the analytics in issue gh-150411, it should be a gc_generation.count update during the cyclic object allocation triggered the local allocation count migrated to young generation. Meantime we try to read the gc_generation.count without sync caused the race.

cpython/Python/gc_free_threading.c

Lines 2017 to 2037 in c714b56

    
           static void 
        
           record_allocation(PyThreadState *tstate) 
        
           { 
        
               struct _gc_thread_state *gc = &((_PyThreadStateImpl *)tstate)->gc; 
        
               // We buffer the allocation count to avoid the overhead of atomic 
        
               // operations for every allocation. 
        
               gc->alloc_count++; 
        
               if (gc->alloc_count >= LOCAL_ALLOC_COUNT_THRESHOLD) { 
        
                   // TODO: Use Py_ssize_t for the generation count. 
        
                   GCState *gcstate = &tstate->interp->gc; 
        
                   _Py_atomic_add_int(&gcstate->young.count, (int)gc->alloc_count); 
        
                   gc->alloc_count = 0; 
        
                   if (gc_should_collect(gcstate) && 
        
                       !_Py_atomic_load_int_relaxed(&gcstate->collecting)) 
        
                   { 
        
                       _Py_ScheduleGC(tstate); 
        
                   } 
        
               } 
        
           }

I find this problem during proposing gh-150356. So they have similar reproduce way.

Proposed Changes

I added an atomic relax load guard for the gc_generation.count.
It was protected in other places expect current one.

Same concerns as #150356 (comment)

CC @nascheme

Does gcstate->old[0].count need to be atomic here? I think it is only ever written under stw so it is unnecessary.

Hi @kumaraditya303 ,

Thanks very much for your review! ❤️
Yes, I searched the usage of count and find that only the young.count is updated without STW guard.
old[generation].count are all protected by STW.
So we only need to make young.count use atomic action and old[generation].count read is safe.

Hi @pablogsal and @kumaraditya303 ,

Thanks very much for your review and suggestions!

I have updated the patch with only atomic for young.count.
The old[generation].count write is protected by STW.
The young.count is a frequently updated counter and in this patch we only read its snapshot.
So I think the result is consistent since we only synced one variable.

Please correct me if I made any mistake or misunderstanding.

Wish you a good day! 🌞

…thon#150413)

Thanks @LindaSummer for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

Thanks @LindaSummer for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

GH-153787 is a backport of this pull request to the 3.14 branch.

GH-153788 is a backport of this pull request to the 3.15 branch.

…H-150413) (#153788) gh-150411: fix `gc_generation.count` race in free-threading (GH-150413) (cherry picked from commit 12af26d) Co-authored-by: Edward Xu <xuxiangad@gmail.com>

…H-150413) (GH-153787) (cherry picked from commit 12af26d) Co-authored-by: Edward Xu <xuxiangad@gmail.com>

LindaSummer requested a review from pablogsal as a code owner May 25, 2026 16:18

bedevere-app Bot mentioned this pull request May 25, 2026

Data race between record_allocation and gc_get_count_impl #150411

Closed

bedevere-app Bot added the awaiting review label May 25, 2026

kumaraditya303 reviewed Jun 4, 2026

View reviewed changes

LindaSummer added 3 commits June 4, 2026 23:33

fix gc_generation.count race

c916f01

add blurb

b2e9bca

remove useless atomic

639cc24

LindaSummer force-pushed the gc_count_tsan branch from b71383e to 639cc24 Compare June 4, 2026 15:50

kumaraditya303 reviewed Jun 6, 2026

View reviewed changes

Comment thread Lib/test/test_free_threading/test_gc.py Outdated

Update Lib/test/test_free_threading/test_gc.py

4ab7a60

kumaraditya303 approved these changes Jun 6, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Jun 6, 2026

kumaraditya303 enabled auto-merge (squash) June 6, 2026 16:36

kumaraditya303 merged commit 12af26d into python:main Jun 6, 2026
59 checks passed

bedevere-app Bot removed the awaiting merge label Jun 6, 2026

philthompson10 pushed a commit to philthompson10/cpython that referenced this pull request Jun 17, 2026

pythongh-150411 : fix gc_generation.count race in free-threading (py…

fa2eb1e

…thon#150413)

nascheme mentioned this pull request Jul 14, 2026

Backport free-threading memory-safety fixes to 3.14 and 3.15 #153714

Closed

nascheme added needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Jul 16, 2026

nascheme added the topic-free-threading label Jul 16, 2026

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jul 16, 2026

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Jul 16, 2026

nascheme pushed a commit that referenced this pull request Jul 17, 2026

[3.14] gh-150411 : fix gc_generation.count race in free-threading (G…

46dbee9

…H-150413) (GH-153787) (cherry picked from commit 12af26d) Co-authored-by: Edward Xu <xuxiangad@gmail.com>

-                                       gcstate->old[0].count,
-                                       gcstate->old[1].count);
+                                       _Py_atomic_load_int_relaxed(&gcstate->young.count),
+                                       _Py_atomic_load_int_relaxed(&gcstate->old[0].count),

gh-150411: fix `gc_generation.count` race by LindaSummer · Pull Request #150413 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-150411: fix gc_generation.count race - #150413

gh-150411: fix gc_generation.count race#150413
kumaraditya303 merged 4 commits into
python:mainfrom
LindaSummer:gc_count_tsan

LindaSummer commented May 25, 2026

Uh oh!

pablogsal commented May 25, 2026

Uh oh!

kumaraditya303 Jun 4, 2026

Uh oh!

LindaSummer Jun 4, 2026

Uh oh!

LindaSummer commented Jun 4, 2026

Uh oh!

Uh oh!

Uh oh!

miss-islington-app Bot commented Jul 16, 2026

Uh oh!

miss-islington-app Bot commented Jul 16, 2026

Uh oh!

bedevere-app Bot commented Jul 16, 2026

Uh oh!

bedevere-app Bot commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	static void
	record_allocation(PyThreadState *tstate)
	{
	struct _gc_thread_state gc = &((_PyThreadStateImpl )tstate)->gc;

	// We buffer the allocation count to avoid the overhead of atomic
	// operations for every allocation.
	gc->alloc_count++;
	if (gc->alloc_count >= LOCAL_ALLOC_COUNT_THRESHOLD) {
	// TODO: Use Py_ssize_t for the generation count.
	GCState *gcstate = &tstate->interp->gc;
	_Py_atomic_add_int(&gcstate->young.count, (int)gc->alloc_count);
	gc->alloc_count = 0;

	if (gc_should_collect(gcstate) &&
	!_Py_atomic_load_int_relaxed(&gcstate->collecting))
	{
	_Py_ScheduleGC(tstate);
	}
	}
	}

gh-150411: fix `gc_generation.count` race by LindaSummer · Pull Request #150413 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-150411: fix gc_generation.count race - #150413

gh-150411: fix gc_generation.count race#150413kumaraditya303 merged 4 commits intopython:mainpython/cpython:mainfrom LindaSummer:gc_count_tsanLindaSummer/cpython:gc_count_tsanCopy head branch name to clipboard

Conversation

LindaSummer commented May 25, 2026

Issue

Root Cause

Proposed Changes

Uh oh!

pablogsal commented May 25, 2026

Uh oh!

kumaraditya303 Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

LindaSummer Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

LindaSummer commented Jun 4, 2026

Uh oh!

Uh oh!

Uh oh!

miss-islington-app Bot commented Jul 16, 2026

Uh oh!

miss-islington-app Bot commented Jul 16, 2026

Uh oh!

bedevere-app Bot commented Jul 16, 2026

Uh oh!

bedevere-app Bot commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gh-150411: fix gc_generation.count race#150413
kumaraditya303 merged 4 commits into
python:mainfrom
LindaSummer:gc_count_tsan