[3.11] gh-135034: Normalize link targets in tarfile, add `os.path.realpath(strict='allow_missing')` (GH-135037) by Yhg1s · Pull Request #135068 · python/cpython · GitHub

Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.
(cherry picked from commit 3612d8f)

Co-authored-by: Łukasz Langa lukasz@langa.pl
Co-authored-by: Petr Viktorin encukou@gmail.com
Co-authored-by: Seth Michael Larson seth@python.org
Co-authored-by: Adam Turner 9087854+AA-Turner@users.noreply.github.com
Co-authored-by: Serhiy Storchaka storchaka@gmail.com

Issue: Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

📚 Documentation preview 📚: https://cpython-previews--135068.org.readthedocs.build/

…th.realpath(strict='allow_missing')` (pythonGH-135037) Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517. (cherry picked from commit 3612d8f) (cherry picked from commit c358142) Co-authored-by: Łukasz Langa <lukasz@langa.pl> Signed-off-by: Łukasz Langa <lukasz@langa.pl> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Seth Michael Larson <seth@python.org> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

security backports.

non-strict test for unreadable symlinks, whch isn't supported before 3.13.

After this update, I am getting the following test failure:

======================================================================
ERROR: test_realpath_limit_attack (test.test_tarfile.TestExtractionFilters.test_realpath_limit_attack) [fully_trusted]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3669, in test_realpath_limit_attack
    with (self.subTest('fully_trusted'),
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3458, in check_context
    self.expected_paths = set(self.outerdir.glob('**/*'))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 958, in glob
    for p in selector.select_from(self):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 411, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  [Previous line repeated 14 more times]
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 395, in _iterate_directories
    entry_is_dir = entry.is_dir(follow_symlinks=False)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 78] File name too long: '/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/build/amd64/build/test_python_22904æ/@test_22904_tmpæ-tardir/outerdir/dest/ddddddddddddddddddd
dddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddddd
ddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddd
dddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddd
ddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/lllllllllllllll
lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
llllllllllllllllllllllllllllllll'

Interestingly, it's just 3.11.13 that's failing - both 3.9.23 and 3.13.4 (and all other supported versions we are running in our internal buildbot) pass. 3.13 has a very different pathlib implementation, but 3.9 is pretty similar so I investigated the differences, and when I print the entires from here (_iterate_directories 3.9 / 3.11):

            with scandir(parent_path) as scandir_it:
                entries = list(scandir_it)

I get very different results:
3.9:

[<DirEntry 'newfile'>, <DirEntry 'flag'>, <DirEntry 'dest'>]
[<DirEntry 'a'>, <DirEntry 'flaglink'>, <DirEntry 'escape'>]

3.11:

[<DirEntry 'dest'>, <DirEntry 'flag'>, <DirEntry 'newfile'>]
[<DirEntry 'a'>, <DirEntry 'escape'>, <DirEntry 'flaglink'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'b'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'c'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'e'>]
[<DirEntry 'f'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'g'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'h'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'i'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'j'>]
[<DirEntry 'k'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'l'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'm'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'n'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'o'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'p'>]
[<DirEntry 'lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
lllllllllllllllllllllllllllllllllllllllllllllllllllllllllll'>]

Also, the scandir argument is different:
3.9: <built-in function scandir>
3.11: <function Path._scandir at 0x7fed9bd923e0>
so I guess that those implementations behave differently (the 3.11 one returns much more entires), which causes the failure?

This is on Oracle Solaris.

Ah! We got the same test failure in 3.10, so I hotfixed it there. Try applying the hack to 3.11: dff62a1

It's just a test failure: if the filter is fully_trusted, this archive unpacks to a very interesting directory tree -- one that this version of Path.glob can't handle. And we call glob in the tests to get data for the assertions.

Oh, that fixed it. Thanks!

(i didn't realize that this difference between 3.9 and 3.11 patch is what causes the issue)

Hello, sorry to ask directly here but i haven't understood how 3.10 & 3.11 are affected by these CVEs, since the description of each CVE states that ONLY versions after 3.12 are affected, since the extraction filters were first introduced in 3.12
Thank you in advance

yeah, looks like the CVE text needs an update :(
cc @sethmlarson
The feature was backported to 3.8.17, 3.9.17, 3.10.12, 3.11.4

@encukou Gotcha, I didn't realize that filtering had been backported, I'll update the prose description in each document. The affectedness of the CVEs is correct thankfully :)

Updated the CVE records and sent a correction to security-announce@python.org.

ambv and others added 2 commits June 3, 2025 13:35

Fix test failure in test_tarfile, remove tests that aren't suitable for

698a6d4

security backports.

Yhg1s requested a review from ethanfurman as a code owner June 3, 2025 11:45

bedevere-app Bot added the awaiting core review label Jun 3, 2025

This was referenced Jun 3, 2025

gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') #135037

Merged

Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

Closed

Yhg1s added 5 commits June 3, 2025 14:10

Fix a few tests, and a glaring logic error in the posixpath backport.

3196554

Stop using :cve:, which doesn't work in 3.11 and earlier.

8ab383f

Fix test_tarfile (for not exporting LinkFallbackError) and remove the

6d4d4d6

non-strict test for unreadable symlinks, whch isn't supported before 3.13.

Fix Windows test_ntpath failures.

630ffa3

Fix merge error in backport.

28d8d22

Yhg1s requested review from ambv, encukou and pablogsal June 3, 2025 13:42

Remove backport of a test that won't pass under 3.11.

92a788a

encukou approved these changes Jun 3, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Jun 3, 2025

ambv merged commit 4633f3f into python:3.11 Jun 3, 2025

bedevere-app Bot removed the awaiting merge label Jun 3, 2025

Yhg1s deleted the backport-c358142-3.11 branch August 13, 2026 12:52

[3.11] gh-135034: Normalize link targets in tarfile, add `os.path.realpath(strict='allow_missing')` (GH-135037) by Yhg1s · Pull Request #135068 · python/cpython · GitHub

Uh oh!

Uh oh!

[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037) - #135068

[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037)#135068
ambv merged 8 commits into
python:3.11from
Yhg1s:backport-c358142-3.11

Yhg1s commented Jun 3, 2025 •
edited
Loading

Uh oh!

Uh oh!

kulikjak commented Jun 4, 2025

Uh oh!

encukou commented Jun 4, 2025

Uh oh!

kulikjak commented Jun 4, 2025 •
edited
Loading

Uh oh!

Uh oh!

DimNik9 commented Jun 5, 2025

Uh oh!

encukou commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[3.11] gh-135034: Normalize link targets in tarfile, add `os.path.realpath(strict='allow_missing')` (GH-135037) by Yhg1s · Pull Request #135068 · python/cpython · GitHub

Uh oh!

Uh oh!

[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037) - #135068

[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037)#135068ambv merged 8 commits intopython:3.11python/cpython:3.11from Yhg1s:backport-c358142-3.11Yhg1s/cpython:backport-c358142-3.11Copy head branch name to clipboard

Conversation

Yhg1s commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kulikjak commented Jun 4, 2025

Uh oh!

encukou commented Jun 4, 2025

Uh oh!

kulikjak commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DimNik9 commented Jun 5, 2025

Uh oh!

encukou commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

[3.11] gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') (GH-135037)#135068
ambv merged 8 commits into
python:3.11from
Yhg1s:backport-c358142-3.11

Yhg1s commented Jun 3, 2025 •
edited
Loading

kulikjak commented Jun 4, 2025 •
edited
Loading