gh-102304: Move _Py_RefTotal to _PyRuntimeState by ericsnowcurrently · Pull Request #102543 · python/cpython · GitHub

The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

Issue: Move _Py_RefTotal to PyInterpreterState #102304

CC @kumaraditya303

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit 2fb3259 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All the refleak buildbots are happy with this PR.

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Gentoo Non-Debug with X 3.x has failed when building commit ad77d16.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/58/builds/4078) and take a look at the build logs.
Check if the failure is related to this commit (ad77d16) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/58/builds/4078

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 46, done.        
remote: Counting objects:   2% (1/46)        
remote: Counting objects:   4% (2/46)        
remote: Counting objects:   6% (3/46)        
remote: Counting objects:   8% (4/46)        
remote: Counting objects:  10% (5/46)        
remote: Counting objects:  13% (6/46)        
remote: Counting objects:  15% (7/46)        
remote: Counting objects:  17% (8/46)        
remote: Counting objects:  19% (9/46)        
remote: Counting objects:  21% (10/46)        
remote: Counting objects:  23% (11/46)        
remote: Counting objects:  26% (12/46)        
remote: Counting objects:  28% (13/46)        
remote: Counting objects:  30% (14/46)        
remote: Counting objects:  32% (15/46)        
remote: Counting objects:  34% (16/46)        
remote: Counting objects:  36% (17/46)        
remote: Counting objects:  39% (18/46)        
remote: Counting objects:  41% (19/46)        
remote: Counting objects:  43% (20/46)        
remote: Counting objects:  45% (21/46)        
remote: Counting objects:  47% (22/46)        
remote: Counting objects:  50% (23/46)        
remote: Counting objects:  52% (24/46)        
remote: Counting objects:  54% (25/46)        
remote: Counting objects:  56% (26/46)        
remote: Counting objects:  58% (27/46)        
remote: Counting objects:  60% (28/46)        
remote: Counting objects:  63% (29/46)        
remote: Counting objects:  65% (30/46)        
remote: Counting objects:  67% (31/46)        
remote: Counting objects:  69% (32/46)        
remote: Counting objects:  71% (33/46)        
remote: Counting objects:  73% (34/46)        
remote: Counting objects:  76% (35/46)        
remote: Counting objects:  78% (36/46)        
remote: Counting objects:  80% (37/46)        
remote: Counting objects:  82% (38/46)        
remote: Counting objects:  84% (39/46)        
remote: Counting objects:  86% (40/46)        
remote: Counting objects:  89% (41/46)        
remote: Counting objects:  91% (42/46)        
remote: Counting objects:  93% (43/46)        
remote: Counting objects:  95% (44/46)        
remote: Counting objects:  97% (45/46)        
remote: Counting objects: 100% (46/46)        
remote: Counting objects: 100% (46/46), done.        
remote: Compressing objects:   5% (1/18)        
remote: Compressing objects:  11% (2/18)        
remote: Compressing objects:  16% (3/18)        
remote: Compressing objects:  22% (4/18)        
remote: Compressing objects:  27% (5/18)        
remote: Compressing objects:  33% (6/18)        
remote: Compressing objects:  38% (7/18)        
remote: Compressing objects:  44% (8/18)        
remote: Compressing objects:  50% (9/18)        
remote: Compressing objects:  55% (10/18)        
remote: Compressing objects:  61% (11/18)        
remote: Compressing objects:  66% (12/18)        
remote: Compressing objects:  72% (13/18)        
remote: Compressing objects:  77% (14/18)        
remote: Compressing objects:  83% (15/18)        
remote: Compressing objects:  88% (16/18)        
remote: Compressing objects:  94% (17/18)        
remote: Compressing objects: 100% (18/18)        
remote: Compressing objects: 100% (18/18), done.        
remote: Total 24 (delta 22), reused 6 (delta 6), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'ad77d16a6252c2e616bf41b981a6d919c1122b4d'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at ad77d16a62 gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
Switched to and reset branch 'main'

configure: WARNING: i686-pc-linux-gnu/gcc is not supported
configure: WARNING:

Platform "i686-pc-linux-gnu" with compiler "gcc" is not supported by the
CPython core team, see https://peps.python.org/pep-0011/ for more information.


Objects/obmalloc.c:775:1: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  775 | arena_map_get(pymem_block *p, int create)
      | ^~~~~~~~~~~~~
In file included from ./Include/Python.h:35,
                 from Python/pystate.c:4:
Python/pystate.c: In function ‘_PyRuntimeState_Fini’:
Python/pystate.c:486:33: error: ‘struct _py_object_runtime_state’ has no member named ‘reftotal’
  486 |     assert(runtime->object_state.reftotal == 0);
      |                                 ^
make: *** [Makefile:2583: Python/pystate.o] Error 1

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:2669: clean-retain-profile] Error 1 (ignored)

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM Raspbian 3.x has failed when building commit ad77d16.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/424/builds/3539) and take a look at the build logs.
Check if the failure is related to this commit (ad77d16) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/424/builds/3539

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 46, done.        
remote: Counting objects:   2% (1/46)        
remote: Counting objects:   4% (2/46)        
remote: Counting objects:   6% (3/46)        
remote: Counting objects:   8% (4/46)        
remote: Counting objects:  10% (5/46)        
remote: Counting objects:  13% (6/46)        
remote: Counting objects:  15% (7/46)        
remote: Counting objects:  17% (8/46)        
remote: Counting objects:  19% (9/46)        
remote: Counting objects:  21% (10/46)        
remote: Counting objects:  23% (11/46)        
remote: Counting objects:  26% (12/46)        
remote: Counting objects:  28% (13/46)        
remote: Counting objects:  30% (14/46)        
remote: Counting objects:  32% (15/46)        
remote: Counting objects:  34% (16/46)        
remote: Counting objects:  36% (17/46)        
remote: Counting objects:  39% (18/46)        
remote: Counting objects:  41% (19/46)        
remote: Counting objects:  43% (20/46)        
remote: Counting objects:  45% (21/46)        
remote: Counting objects:  47% (22/46)        
remote: Counting objects:  50% (23/46)        
remote: Counting objects:  52% (24/46)        
remote: Counting objects:  54% (25/46)        
remote: Counting objects:  56% (26/46)        
remote: Counting objects:  58% (27/46)        
remote: Counting objects:  60% (28/46)        
remote: Counting objects:  63% (29/46)        
remote: Counting objects:  65% (30/46)        
remote: Counting objects:  67% (31/46)        
remote: Counting objects:  69% (32/46)        
remote: Counting objects:  71% (33/46)        
remote: Counting objects:  73% (34/46)        
remote: Counting objects:  76% (35/46)        
remote: Counting objects:  78% (36/46)        
remote: Counting objects:  80% (37/46)        
remote: Counting objects:  82% (38/46)        
remote: Counting objects:  84% (39/46)        
remote: Counting objects:  86% (40/46)        
remote: Counting objects:  89% (41/46)        
remote: Counting objects:  91% (42/46)        
remote: Counting objects:  93% (43/46)        
remote: Counting objects:  95% (44/46)        
remote: Counting objects:  97% (45/46)        
remote: Counting objects: 100% (46/46)        
remote: Counting objects: 100% (46/46), done.        
remote: Compressing objects:   5% (1/18)        
remote: Compressing objects:  11% (2/18)        
remote: Compressing objects:  16% (3/18)        
remote: Compressing objects:  22% (4/18)        
remote: Compressing objects:  27% (5/18)        
remote: Compressing objects:  33% (6/18)        
remote: Compressing objects:  38% (7/18)        
remote: Compressing objects:  44% (8/18)        
remote: Compressing objects:  50% (9/18)        
remote: Compressing objects:  55% (10/18)        
remote: Compressing objects:  61% (11/18)        
remote: Compressing objects:  66% (12/18)        
remote: Compressing objects:  72% (13/18)        
remote: Compressing objects:  77% (14/18)        
remote: Compressing objects:  83% (15/18)        
remote: Compressing objects:  88% (16/18)        
remote: Compressing objects:  94% (17/18)        
remote: Compressing objects: 100% (18/18)        
remote: Compressing objects: 100% (18/18), done.        
remote: Total 24 (delta 22), reused 6 (delta 6), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'ad77d16a6252c2e616bf41b981a6d919c1122b4d'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at ad77d16a62 gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
Switched to and reset branch 'main'

Objects/obmalloc.c:775:1: warning: ‘always_inline’ function might not be inlinable [-Wattributes]
  775 | arena_map_get(pymem_block *p, int create)
      | ^~~~~~~~~~~~~
In file included from ./Include/Python.h:35,
                 from Python/pystate.c:4:
Python/pystate.c: In function ‘_PyRuntimeState_Fini’:
Python/pystate.c:486:33: error: ‘struct _py_object_runtime_state’ has no member named ‘reftotal’
  486 |     assert(runtime->object_state.reftotal == 0);
      |                                 ^
make: *** [Makefile:2579: Python/pystate.o] Error 1
make: *** Waiting for unfinished jobs....

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make: [Makefile:2665: clean-retain-profile] Error 1 (ignored)

gh-102846 should take care of those buildbots.

Some debug-only code slipped in with gh-102543. #102304

The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState. Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead. python#102304

Some debug-only code slipped in with pythongh-102543. python#102304

The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState. Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead. python#102304

Some debug-only code slipped in with pythongh-102543. python#102304

ericsnowcurrently added the skip news label Mar 8, 2023

bedevere-bot mentioned this pull request Mar 8, 2023

Move _Py_RefTotal to PyInterpreterState #102304

Closed

bedevere-bot added the awaiting core review label Mar 8, 2023

ericsnowcurrently mentioned this pull request Mar 8, 2023

gh-102304: Move the Total Refcount to PyInterpreterState #102545

Merged

ericsnowcurrently added 9 commits March 9, 2023 09:56

_Py_RefTotal -> REFTOTAL.

005a143

Separate the internal helper functions.

b5c783b

Drop the Py_BUILD_CORE case.

d286080

Add a blank line.

31a14ed

_Py_GetRefTotal() -> _Py_GetGlobalRefTotal().

8ca3983

Move _Py_RefTotal to _PyRuntimeState.

bff6a6d

Include the legacy total in _Py_GetGlobalRefTotal().

5ab4c4b

Fix get_global_refcount().

cc5a99a

Stop being so clever with PyAPI_FUNC().

f382c4a

ericsnowcurrently force-pushed the consolidate-py-reftotal branch from 8af10a5 to f382c4a Compare March 9, 2023 17:11

ericsnowcurrently added 8 commits March 9, 2023 12:10

Drop unnecessary incr/decr to the reftotal.

38e0860

Preserve the reftotal past runtime finalization.

7dc494d

Avoid double-counting the final reftotal.

136be3a

Factor out sync_legacy_reftotal().

0abec9c

Drop last_legacy_reftotal.

6e4425c

Fix the REFTOTAL macro.

49f1502

_Py_ClearRefTotal() -> _Py_FinalizeRefTotal().

f874ae0

Add a new global to the ingored list.

2fb3259

ericsnowcurrently added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 17, 2023

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 17, 2023

Fix _Py_FinalizeRefTotal().

2a11934

ericsnowcurrently marked this pull request as ready for review March 20, 2023 15:21

ericsnowcurrently requested a review from a team as a code owner March 20, 2023 15:21

ericsnowcurrently merged commit ad77d16 into python:main Mar 20, 2023

bedevere-bot removed the awaiting core review label Mar 20, 2023

ericsnowcurrently deleted the consolidate-py-reftotal branch March 20, 2023 16:03

ericsnowcurrently mentioned this pull request Mar 20, 2023

gh-102304: Fix Non-Debug Builds #102846

Merged

ericsnowcurrently added a commit that referenced this pull request Mar 20, 2023

gh-102304 : Fix Non-Debug Builds (gh-102846 )

5c75b7a

Some debug-only code slipped in with gh-102543. #102304

Fidget-Spinner pushed a commit to Fidget-Spinner/cpython that referenced this pull request Mar 27, 2023

pythongh-102304 : Fix Non-Debug Builds (pythongh-102846 )

76fb37b

Some debug-only code slipped in with pythongh-102543. python#102304

warsaw pushed a commit to warsaw/cpython that referenced this pull request Apr 11, 2023

pythongh-102304 : Fix Non-Debug Builds (pythongh-102846 )

4f1afcc

Some debug-only code slipped in with pythongh-102543. python#102304

gh-102304: Move _Py_RefTotal to _PyRuntimeState by ericsnowcurrently · Pull Request #102543 · python/cpython · GitHub

Repository navigation

gh-102304: Move _Py_RefTotal to _PyRuntimeState - #102543

gh-102304: Move _Py_RefTotal to _PyRuntimeState#102543
ericsnowcurrently merged 18 commits into
python:mainfrom
ericsnowcurrently:consolidate-py-reftotal

ericsnowcurrently commented Mar 8, 2023 •
edited
Loading

Uh oh!

Uh oh!

ericsnowcurrently commented Mar 8, 2023

Uh oh!

bedevere-bot commented Mar 17, 2023

Uh oh!

ericsnowcurrently commented Mar 20, 2023

Uh oh!

bedevere-bot commented Mar 20, 2023

Uh oh!

bedevere-bot commented Mar 20, 2023

Uh oh!

ericsnowcurrently commented Mar 20, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gh-102304: Move _Py_RefTotal to _PyRuntimeState by ericsnowcurrently · Pull Request #102543 · python/cpython · GitHub

Repository navigation

gh-102304: Move _Py_RefTotal to _PyRuntimeState - #102543

gh-102304: Move _Py_RefTotal to _PyRuntimeState#102543ericsnowcurrently merged 18 commits intopython:mainpython/cpython:mainfrom ericsnowcurrently:consolidate-py-reftotalericsnowcurrently/cpython:consolidate-py-reftotalCopy head branch name to clipboard

Conversation

ericsnowcurrently commented Mar 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ericsnowcurrently commented Mar 8, 2023

Uh oh!

bedevere-bot commented Mar 17, 2023

Uh oh!

ericsnowcurrently commented Mar 20, 2023

Uh oh!

bedevere-bot commented Mar 20, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Mar 20, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

ericsnowcurrently commented Mar 20, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gh-102304: Move _Py_RefTotal to _PyRuntimeState#102543
ericsnowcurrently merged 18 commits into
python:mainfrom
ericsnowcurrently:consolidate-py-reftotal

ericsnowcurrently commented Mar 8, 2023 •
edited
Loading