FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-148286: Fix UB in `ZstdDecompressor.unused_data` when a frame is decompressed in one call by StanFromIreland · Pull Request #153258 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-148286: Fix UB in ZstdDecompressor.unused_data when a frame is decompressed in one call - #153258

Merged
StanFromIreland merged 2 commits into
python:mainfrom
StanFromIreland:zstd-unused-data
Jul 13, 2026
ConversationCommits (2)ChecksFiles changed
Merged

gh-148286: Fix UB in ZstdDecompressor.unused_data when a frame is decompressed in one call#153258
StanFromIreland merged 2 commits into
python:mainfrom
StanFromIreland:zstd-unused-data

Conversation

StanFromIreland commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

To reproduce, run test_zstd.CompressorTestCase.test_compress_flushblock or under UBSan:

>>> from compression.zstd import ZstdDecompressor, compress
>>> d = ZstdDecompressor()
>>> d.decompress(compress(b'Hi Emma!'))
b'Hi Emma!'
>>> d.unused_data
Modules/_zstd/decompressor.c:598:56: runtime error: applying non-zero offset 18446744073709551615 to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/_zstd/decompressor.c:598:56 
b''
>>>

ZstdDecompressor.unused_data computes self->input_buffer + self->in_begin without checking that input_buffer was ever allocated, so when a frame is fully consumed in a single decompress() call it leaves input_buffer NULL (and in_begin/in_end at their initial state). Reading .unused_data evaluates NULL + ..., which is UB. Although the resulting length of in_end - in_begin == 0 means it accidentally returns the correct b''.

The other unused_data implementations are not affected due to their design.

Also, CC @Rogdham

bedevere-app Bot mentioned this pull request Jul 7, 2026
Open
emmatyping approved these changes Jul 13, 2026
View reviewed changes

emmatyping left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks for fixing this!

StanFromIreland added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Jul 13, 2026
StanFromIreland merged commit adebb68 into python:main Jul 13, 2026
64 checks passed
StanFromIreland deleted the zstd-unused-data branch July 13, 2026 07:02

miss-islington-app Bot commented Jul 13, 2026

Copy link
Copy Markdown

Thanks @StanFromIreland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

miss-islington-app Bot commented Jul 13, 2026

Copy link
Copy Markdown

Sorry, @StanFromIreland, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker adebb68153346043c0671fa5725d269c32cc40e4 3.14

bedevere-app Bot commented Jul 13, 2026

Copy link
Copy Markdown

GH-153644 is a backport of this pull request to the 3.15 branch.

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Jul 13, 2026

miss-islington-app Bot commented Jul 13, 2026

Copy link
Copy Markdown

Sorry, @StanFromIreland, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker adebb68153346043c0671fa5725d269c32cc40e4 3.13

StanFromIreland removed needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Jul 13, 2026

bedevere-app Bot commented Jul 13, 2026

Copy link
Copy Markdown

GH-153645 is a backport of this pull request to the 3.14 branch.

StanFromIreland added a commit that referenced this pull request Jul 13, 2026
[3.14] gh-148286: Fix UB in ZstdDecompressor.unused_data when a fra…
b0842ed 
…me is decompressed in one call (GH-153258) (#153645)

(cherry picked from commit adebb68)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
StanFromIreland added a commit that referenced this pull request Jul 13, 2026
[3.15] gh-148286: Fix UB in ZstdDecompressor.unused_data when a fra…
9108488 
…me is decompressed in one call (GH-153258) (#153644)

(cherry picked from commit adebb68)

Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Emma Smith <emma@emmatyping.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

emmatyping emmatyping approved these changes
AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

Assignees

StanFromIreland

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL