FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

GH-90699: fix ref counting of static immortal strings by kumaraditya303 · Pull Request #94850 · python/cpython · GitHub

/ cpython Public

GH-90699: fix ref counting of static immortal strings - #94850

Merged
corona10 merged 4 commits into
python:mainfrom
kumaraditya303:cached-strs
Jul 20, 2022
Merged

GH-90699: fix ref counting of static immortal strings#94850
corona10 merged 4 commits into
python:mainfrom
kumaraditya303:cached-strs

Conversation

kumaraditya303 commented Jul 14, 2022
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Contributor

Without this they can cause negative refcount in _Py_RefTotal.

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit f530250 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 14, 2022
kumaraditya303 added the needs backport to 3.11 only security fixes label Jul 14, 2022

corona10 commented Jul 14, 2022
edited
Loading

Copy link
Copy Markdown
Member

Thanks for the work, Would you like to provide a reproducible way for detecting leaks?

Copy link
Copy Markdown
Contributor Author

Thanks for the work, Would you like to provide a reproducible way for detecting leaks?

The following script causes negative ref count:

for i in range(1000):
    a = repr(False)

Output:

@kumaraditya303 ➜ /workspaces/cpython (main ✗) $ ./python -X showrefcount main.py 
[-1001 refs, 0 blocks]

kumaraditya303 requested a review from corona10 July 14, 2022 12:59

Copy link
Copy Markdown
Contributor Author

@corona10: The following script causes negative refcount with TextIOWrapper:

# main.py
from io import TextIOWrapper

a = TextIOWrapper(open('main.py','rb'))
for i in a:
    pass
a.close()

Output:

@kumaraditya303 ➜ /workspaces/cpython (main ✗) $ ./python -X showrefcount main.py 
[-2 refs, 0 blocks]

corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

lgtm

I reproduced the issue with the attached scripts and issues are solved by @kumaraditya303 's patch :)

Thanks for the work!

corona10 left a comment
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@kumaraditya303

Sorry Kumar, Looks like leaks exist through this patch.
PTAL

With patch

beginning 6 repetitions
123456
......
test_io leaked [2, 2, 2] references, sum=6
test_io failed (reference leak) in 3 min 23 sec

== Tests result: FAILURE ==

1 test failed:
    test_io

Total duration: 3 min 23 sec
Tests result: FAILURE

Without Patch

Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 2.90 Run tests sequentially
0:00:00 load avg: 2.90 [1/1] test_io
beginning 6 repetitions
123456
......
test_io passed in 3 min 23 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 3 min 23 sec
Tests result: SUCCESS

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Copy link
Copy Markdown
Contributor Author

Looking, I wonder why only two bots failed if there are refleaks.

Copy link
Copy Markdown
Contributor Author

It is also leaking at my local environment :(

Yeah, I am also able to reproduce, bisecting which one is causing leak

Copy link
Copy Markdown
Contributor Author

I am getting refleak on main branch too, test.test_io.CTextIOWrapperTest.test_reconfigure_locale is failing.

kumaraditya303 commented Jul 15, 2022
edited
Loading

Copy link
Copy Markdown
Contributor Author

Refleak was fixed by #94858

kumaraditya303 requested a review from corona10 July 18, 2022 15:16
kumaraditya303 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 18, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit 0a76abd 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 18, 2022

Copy link
Copy Markdown
Contributor Author

Refleak buildbots are failing on main too, see #94979

corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM
Thank you, Kumar!

corona10 merged commit 1834133 into python:main Jul 20, 2022

Copy link
Copy Markdown
Contributor

Thanks @kumaraditya303 for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jul 20, 2022
…-94850)

(cherry picked from commit 1834133)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

Copy link
Copy Markdown

GH-95037 is a backport of this pull request to the 3.11 branch.

bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 20, 2022
kumaraditya303 deleted the cached-strs branch July 20, 2022 06:28

Copy link
Copy Markdown
Contributor Author

Thanks for the review @corona10!

miss-islington added a commit that referenced this pull request Jul 20, 2022
(cherry picked from commit 1834133)

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants


Back | FazBrowse Home | New Git URL