bpo-39068 guard _b85chars2 initialization by drmonkeysee · Pull Request #17627 · python/cpython · GitHub

Under multi-threading scenarios a race condition may occur where a thread sees an initialized _b85chars table but an uninitialized _b85chars2 table due to the guard only checking the first table.

This causes an exception like:

  File "/usr/lib/python3.6/base64.py", line 434, in b85encode
    return _85encode(b, _b85chars, _b85chars2, pad),
  File "/usr/lib/python3.6/base64.py", line 294, in _85encode
    for word in words],
  File "/usr/lib/python3.6/base64.py", line 294, in <listcomp>
    for word in words],
 "TypeError: 'NoneType' object is not subscriptable

https://bugs.python.org/issue39068

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@drmonkeysee

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

It is enough to check if _b85chars2 is None here.

These checks are cheap, but they are performed at every b85encode() call. Lesser checks is better.

a85encode() should be updated too. And may be other functions if they initialize two or more globals.

done. visually scanned the file for other cases but only found the additional one in a85encode().

Please add a NEWS entry. Since it is your first contribution, add also your name in Misc/ACKS.

Thank you for your contribution @drmonkeysee. Sorry for the delay, this ЗК fell out of my window of attention.

Thanks @drmonkeysee for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

Thanks @drmonkeysee for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

There was a race condition in base64 in lazy initialization of multiple globals. (cherry picked from commit 9655434) Co-authored-by: Brandon Stansbury <brandonrstansbury@gmail.com>

GH-24020 is a backport of this pull request to the 3.9 branch.

Sorry, @drmonkeysee and @serhiy-storchaka, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 9655434cca5dfbea97bf6d355aec028e840b289c 3.8

There was a race condition in base64 in lazy initialization of multiple globals.. (cherry picked from commit 9655434) Co-authored-by: Brandon Stansbury <brandonrstansbury@gmail.com>

GH-24022 is a backport of this pull request to the 3.8 branch.

There was a race condition in base64 in lazy initialization of multiple globals. (cherry picked from commit 9655434) Co-authored-by: Brandon Stansbury <brandonrstansbury@gmail.com>

Thanks @drmonkeysee for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

There was a race condition in base64 in lazy initialization of multiple globals. (cherry picked from commit 9655434) Co-authored-by: Brandon Stansbury <brandonrstansbury@gmail.com>

GH-24051 is a backport of this pull request to the 3.9 branch.

There was a race condition in base64 in lazy initialization of multiple globals. (cherry picked from commit 9655434) Co-authored-by: Brandon Stansbury <brandonrstansbury@gmail.com>

There was a race condition in base64 in lazy initialization of multiple globals.

guard _b85chars2 table

04967d1

the-knights-who-say-ni added the CLA not signed label Dec 16, 2019

bedevere-bot added the awaiting review label Dec 16, 2019

Commod0re approved these changes Dec 16, 2019

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Dec 16, 2019

camhawkins approved these changes Dec 16, 2019

View reviewed changes

drmonkeysee changed the title guard _b85chars2 table bpo-39068 guard _b85chars2 table Dec 16, 2019

drmonkeysee changed the title bpo-39068 guard _b85chars2 table bpo-39068 guard _b85chars2 initialization Dec 16, 2019

IEarle approved these changes Dec 16, 2019

View reviewed changes

serhiy-storchaka reviewed Dec 16, 2019

View reviewed changes

only check latter var and fix a85 as well

bf5a325

the-knights-who-say-ni added CLA signed and removed CLA not signed labels Dec 17, 2019

drmonkeysee added 2 commits January 6, 2020 10:07

add news and ack entries

9e86e22

rename file to match time the bug was filed

6562ee4

serhiy-storchaka approved these changes Dec 31, 2020

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Dec 31, 2020

serhiy-storchaka merged commit 9655434 into python:master Dec 31, 2020

bedevere-bot removed the awaiting merge label Dec 31, 2020

serhiy-storchaka added needs backport to 3.8 type-bug An unexpected behavior, bug, or error labels Dec 31, 2020

bedevere-bot removed the needs backport to 3.9 label Dec 31, 2020

miss-islington assigned serhiy-storchaka Dec 31, 2020

bedevere-bot removed the needs backport to 3.8 label Dec 31, 2020

serhiy-storchaka removed their assignment Dec 31, 2020

serhiy-storchaka added the needs backport to 3.9 label Jan 1, 2021

bedevere-bot removed the needs backport to 3.9 label Jan 1, 2021

drmonkeysee deleted the base85-race-condition-fix branch January 4, 2021 19:28

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021

bpo-39068: Fix race condition in base64 (pythonGH-17627 )

287d230

There was a race condition in base64 in lazy initialization of multiple globals.

		@@ -430,6 +430,7 @@ def b85encode(b, pad=False):
		# if the function is never called
		if _b85chars is None:

bpo-39068 guard _b85chars2 initialization by drmonkeysee · Pull Request #17627 · python/cpython · GitHub

Repository navigation

bpo-39068 guard _b85chars2 initialization - #17627

bpo-39068 guard _b85chars2 initialization#17627serhiy-storchaka merged 4 commits intopython:masterpython/cpython:masterfrom drmonkeysee:base85-race-condition-fixdrmonkeysee/cpython:base85-race-condition-fixCopy head branch name to clipboard

Conversation

drmonkeysee commented Dec 16, 2019 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

the-knights-who-say-ni commented Dec 16, 2019

Recognized GitHub username

Uh oh!

serhiy-storchaka Dec 16, 2019

Choose a reason for hiding this comment

Uh oh!

drmonkeysee Dec 16, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka commented Jan 5, 2020

Uh oh!

serhiy-storchaka commented Dec 31, 2020

Uh oh!

miss-islington commented Dec 31, 2020

Uh oh!

miss-islington commented Dec 31, 2020

Uh oh!

bedevere-bot commented Dec 31, 2020

Uh oh!

miss-islington commented Dec 31, 2020

Uh oh!

bedevere-bot commented Dec 31, 2020

Uh oh!

miss-islington commented Jan 1, 2021

Uh oh!

bedevere-bot commented Jan 1, 2021

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

bpo-39068 guard _b85chars2 initialization#17627
serhiy-storchaka merged 4 commits into
python:masterfrom
drmonkeysee:base85-race-condition-fix

drmonkeysee commented Dec 16, 2019 •
edited by bedevere-bot
Loading

drmonkeysee Dec 16, 2019 •
edited
Loading