FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-156939: Fix struct.pack('0p', bytes) by vstinner · Pull Request #157071 · python/cpython · GitHub

Repository navigation

gh-156939: Fix struct.pack('0p', bytes) - #157071

Merged
vstinner merged 1 commit into
python:mainfrom
vstinner:struct_empty_pascal
Sep 7, 2026
Merged

vstinner merged 1 commit into
python:mainfrom
vstinner:struct_empty_pascal

Conversation

vstinner commented Sep 7, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

If the Pascal string is empty (size=0), do not write the size prefix. Previously, a NUL byte was written outsize the buffer (buffer overflow).

If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow).
vstinner added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Sep 7, 2026
vstinner changed the title gh-156943: Fix struct.pack('0p', bytes) gh-156939: Fix struct.pack('0p', bytes) Sep 7, 2026

vstinner commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

In Python 3.14 and older, struct.pack('0p', b'abc') writes a NUL byte after the bytes contents. But bytes objects allocates an extra byte for a trailing NUL byte, so the buffer overflow doesn't write arbitrary memory. It's just a silent write which doesn't corrupt anything.

On Python 3.15 and newer, struct.pack() uses PyBytesWriter which doesn't allocate an extra trailing NUL byte for small strings (up to 256 bytes). And so a buffer overflow can write outsize PyBytesWriter.small_buffer. In practice, it should write a NUL byte in PyBytesWriter.obj which is a NULL pointer, so it should not corrupt arbitrary memory.

Well, it's better to avoid a buffer overflow anyway :-)

vstinner commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

Ah, I forgot to mention that the 0p format always produces an empty bytes string:

>>> struct.pack('0p', b'abc')
b''

vstinner commented Sep 7, 2026

Copy link
Copy Markdown
Member Author

In Python 3.14 and older, struct.pack('0p', b'abc') writes a NUL byte after the bytes contents. But bytes objects allocates an extra byte for a trailing NUL byte, so the buffer overflow doesn't write arbitrary memory.

Oh, I forgot that Python 3.13 and 3.14 uses the old internal _PyBytesWriter API. struct.pack('0p', b'abc') uses a small buffer in the writer API. The small buffer is between 10 and 512 bytes, so writing a NUL byte a position 0 doesn't corrupt memory. It's just a silent (ignored) write.

vstinner merged commit 23525c9 into python:main Sep 7, 2026
61 checks passed
vstinner deleted the struct_empty_pascal branch September 7, 2026 19:32

Copy link
Copy Markdown

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

bedevere-app Bot commented Sep 7, 2026

Copy link
Copy Markdown

GH-157129 is a backport of this pull request to the 3.15 branch.

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Sep 7, 2026

bedevere-app Bot commented Sep 7, 2026

Copy link
Copy Markdown

GH-157130 is a backport of this pull request to the 3.14 branch.

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Sep 7, 2026

bedevere-app Bot commented Sep 7, 2026

Copy link
Copy Markdown

GH-157131 is a backport of this pull request to the 3.13 branch.

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Sep 7, 2026
vstinner added a commit that referenced this pull request Sep 7, 2026
gh-156939: Fix struct.pack('0p', bytes) (GH-157071)

If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow). In practice, the write remains into allocated memory
and is silently ignored: no memory is corrupted.
(cherry picked from commit 23525c9)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this pull request Sep 7, 2026
gh-156939: Fix struct.pack('0p', bytes) (GH-157071)

If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow). In practice, the write remains into allocated memory
and is silently ignored: no memory is corrupted.
(cherry picked from commit 23525c9)

Co-authored-by: Victor Stinner <vstinner@python.org>

vstinner commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

On Python 3.14 and older, if the struct.pack() output size is 512 bytes, this bug can lead to a buffer overflow writing one NUL byte into the stack memory. Example of code triggering the overflow:

import struct
size = 512
res = struct.pack(f'{size}s0p', b'x' * size, b'ignored')
print(len(res))

clin1234 pushed a commit to clin1234/cpython that referenced this pull request Sep 12, 2026
If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow). In practice, the write remains into allocated memory
and is silently ignored: no memory is corrupted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL