FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-141311: Avoid assertion in BytesIO readinto by cmaloney · Pull Request #141333 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-141311: Avoid assertion in BytesIO readinto - #141333

Merged
serhiy-storchaka merged 9 commits into
python:mainfrom
cmaloney:bytesio_seek_far
Nov 12, 2025
ConversationCommits (9)ChecksFiles changed
Merged

gh-141311: Avoid assertion in BytesIO readinto#141333
serhiy-storchaka merged 9 commits into
python:mainfrom
cmaloney:bytesio_seek_far

Conversation

cmaloney commented Nov 10, 2025 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Account for when self->pos is equal to PY_SSIZE_T_MAX. The length of read was correctly set to zero but the asserts assumed self->pos couldn't reach PY_SSIZE_T_MAX. Return early to avoid edge cases.

pythongh-141311: Avoid assertion in BytesIO readinto
db81df6 
Account for when self->pos is equal to PY_SSIZE_T_MAX. The length of
read was correctly set to zero but the asserts assumed
self->pos couldn't reach PY_SSIZE_T_MAX. Return early to avoid edge
cases.
bedevere-app Bot mentioned this pull request Nov 10, 2025
Closed
serhiy-storchaka reviewed Nov 10, 2025
View reviewed changes
Comment thread Modules/_io/bytesio.c Outdated
Comment thread Lib/test/test_io/test_memoryio.py Outdated
Comment thread Misc/NEWS.d/next/Library/2025-11-09-18-55-13.gh-issue-141311.qZ3swc.rst Outdated
Simplify by using sys.maxsize
2c84efe
Reference io.BytesIO.readinto directly
d119904
Resolve read_bytes_lock_held and audit other cases
d78cc84 
For `read_bytes_lock_held` the `size` parameter is always set to 0 by calling
code currently if self->pos > self->string_size. I added a range check for
self->pos as an extra safety as codepaths chagne but can remove if it's too much.

`write_bytes_lock_held`: `endpos` is a `size_t` so can hold
`2 * PY_SSIZE_T_MAX`. Value is bounds checked in `resize_buffer_lock_held`.

A number of cases use `scan_eol_lock_held` to move forward. That has code which
checks `self->pos >= self->string_size` and returns `0`. Callsites all seem to
handle that correctly.

cmaloney commented Nov 11, 2025

Copy link
Copy Markdown
Contributor Author

I audited the codepaths which did self->pos + buffer.

For read_bytes_lock_held the size parameter is always set to 0 by calling code currently if self->pos > self->string_size. I added a range check for self->pos as an extra safety as codepaths change but can remove if it's too much.

write_bytes_lock_held: endpos is a size_t so can hold 2 * PY_SSIZE_T_MAX. Value is bounds checked in resize_buffer_lock_held.

A number of cases use scan_eol_lock_held to move forward. That has code which checks self->pos >= self->string_size and returns 0. Callers seem to handle that correctly (early exit or call read_bytes_lock_held)..

serhiy-storchaka reviewed Nov 11, 2025
View reviewed changes
Comment thread Modules/_io/bytesio.c Outdated
Comment thread Modules/_io/bytesio.c Outdated
Comment thread Modules/_io/bytesio.c Outdated
serhiy-storchaka reviewed Nov 11, 2025
View reviewed changes
Comment thread Modules/_io/bytesio.c Outdated
serhiy-storchaka approved these changes Nov 11, 2025
View reviewed changes

serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM. 👍

serhiy-storchaka merged commit 7d54374 into python:main Nov 12, 2025
50 checks passed
serhiy-storchaka added needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes labels Nov 12, 2025

miss-islington-app Bot commented Nov 12, 2025

Copy link
Copy Markdown

Thanks @cmaloney for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app Bot commented Nov 12, 2025

Copy link
Copy Markdown

Thanks @cmaloney for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington-app Bot commented Nov 12, 2025

Copy link
Copy Markdown

Sorry, @cmaloney and @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 7d54374f9c7d91e0ef90c4ad84baf10073cf1d8a 3.13

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 12, 2025
pythongh-141311: Avoid assertion in BytesIO.readinto() (pythonGH-141333)
7151cfb 
Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
(cherry picked from commit 7d54374)

Co-authored-by: Cody Maloney <cmaloney@users.noreply.github.com>

bedevere-app Bot commented Nov 12, 2025

Copy link
Copy Markdown

GH-141457 is a backport of this pull request to the 3.14 branch.

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 12, 2025

serhiy-storchaka commented Nov 12, 2025

Copy link
Copy Markdown
Member

@cmaloney, could you please create a backport to 3.13 manually?

serhiy-storchaka pushed a commit that referenced this pull request Nov 12, 2025
[3.14] gh-141311: Avoid assertion in BytesIO.readinto() (GH-141333) (G…
75b5157 
…H-141457)

Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
(cherry picked from commit 7d54374)

Co-authored-by: Cody Maloney <cmaloney@users.noreply.github.com>
cmaloney deleted the bytesio_seek_far branch November 12, 2025 19:57

bedevere-app Bot commented Nov 12, 2025

Copy link
Copy Markdown

GH-141478 is a backport of this pull request to the 3.13 branch.

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Nov 12, 2025
serhiy-storchaka pushed a commit that referenced this pull request Nov 12, 2025
[3.13] gh-141311: Avoid assertion in BytesIO.readinto() (GH-141333) (G…
7c62bd5 
…H-141478)

Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
(cherry picked from commit 7d54374)
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
pythongh-141311: Avoid assertion in BytesIO.readinto() (pythonGH-141333)
7592d29 
Fix error in assertion which causes failure if pos is equal to PY_SSIZE_T_MAX.
Fix undefined behavior in read() and readinto() if pos is larger that the size
of the underlying buffer.
serhiy-storchaka removed their assignment Jul 15, 2026
matthiasgoergens mentioned this pull request Aug 5, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

serhiy-storchaka serhiy-storchaka approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

Back | FazBrowse Home | New Git URL