gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. by CorvinM · Pull Request #108657 · python/cpython · GitHub

Fixes an exception where sqlite3.iterdump would fail to decode if someone stuffed invalid unicode into a database column marked as TEXT (or alias of, like VARCHAR).
See #108590 for more info

Issue: sqlite3.iterdump() incompatible with binary data #108590

…umns.

…k0pOl.rst Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Thanks for doing this, Corvin; it is a nice solution to this corner case.

I'll make some final adjustments before landing.

@CorvinM, you may also be interested in looking at #108364

Thank you for the speedy reviews and helpful feedback 😄 Let me know if you want me to pick up anything else with this (e.g., backports)

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.12

Sorry, @CorvinM and @erlend-aasland, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 400a1cebc743515e40157ed7af86e48d654290ce 3.11

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

@CorvinM, can you handle the backports? You can use the cherry-picker. See the devguide.

Sure

…EXT columns (pythonGH-108657) Co-authored-by: Erlend E. Aasland <erlend@python.org>. (cherry picked from commit 400a1ce) Co-authored-by: Corvin <corvin@corvin.dev>

GH-108673 is a backport of this pull request to the 3.12 branch.

…EXT columns (pythonGH-108657) Co-authored-by: Erlend E. Aasland <erlend@python.org>. (cherry picked from commit 400a1ce) Co-authored-by: Corvin <corvin@corvin.dev>

GH-108674 is a backport of this pull request to the 3.11 branch.

…lumns (GH-108657) (#108674) (cherry picked from commit 400a1ce)

Reverted per Serhiy's request.

* main: pythongh-108520: Fix bad fork detection in nested multiprocessing use case (python#108568) pythongh-108590: Revert pythongh-108657 (commit 400a1ce) (python#108686) pythongh-108494: Argument Clinic: Document how to generate code that uses the limited C API (python#108584) Document Python build requirements (python#108646) pythongh-101100: Fix Sphinx warnings in the Logging Cookbook (python#108678) Fix typo in multiprocessing docs (python#108666) pythongh-108669: unittest: Fix documentation for TestResult.collectedDurations (python#108670) pythongh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (python#108657) Revert "pythongh-103224: Use the realpath of the Python executable in `test_venv` (pythonGH-103243)" (pythonGH-108667) pythongh-106320: Remove private _Py_ForgetReference() (python#108664) Mention Ellipsis pickling in the docs (python#103660) Revert "Use non alternate name for Kyiv (pythonGH-108533)" (pythonGH-108649) pythongh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (python#108632) pythongh-108455: peg_generator: install two stubs packages before running mypy (python#108637) pythongh-107801: Improve the accuracy of io.IOBase.seek docs (python#108268)

…on#108686) (cherry picked from commit 2a3926f) Reverted per Serhiy's request.

(cherry picked from commit 2a3926f) Reverted per Serhiy's request.

pythongh-108590 : Fix sqlite3.iterdump for invalid unicode in text col…

a689bbd

…umns.

bedevere-bot mentioned this pull request Aug 29, 2023

sqlite3.iterdump() incompatible with binary data #108590

Closed

2 tasks done

CorvinM commented Aug 29, 2023

View reviewed changes

Comment thread Lib/sqlite3/dump.py

CorvinM marked this pull request as ready for review August 29, 2023 22:49

CorvinM requested review from berkerpeksag and erlend-aasland as code owners August 29, 2023 22:49

bedevere-bot added the awaiting review label Aug 29, 2023

📜🤖 Added by blurb_it.

03f41df

erlend-aasland reviewed Aug 30, 2023

View reviewed changes

Comment thread Lib/sqlite3/dump.py Outdated

Comment thread Misc/NEWS.d/next/Library/2023-08-29-22-53-48.gh-issue-108590.6k0pOl.rst Outdated

Comment thread Lib/sqlite3/dump.py Outdated

CorvinM and others added 3 commits August 30, 2023 03:41

Update Misc/NEWS.d/next/Library/2023-08-29-22-53-48.gh-issue-108590.6…

e2939d5

…k0pOl.rst Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Update Lib/sqlite3/dump.py

0643ffc

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Address comments

608d919

erlend-aasland reviewed Aug 30, 2023

View reviewed changes

Comment thread Lib/sqlite3/dump.py Outdated

Use contextmanager

cc7fba8

erlend-aasland approved these changes Aug 30, 2023

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting review labels Aug 30, 2023

Style nits and a comment

c1b730d

erlend-aasland enabled auto-merge (squash) August 30, 2023 08:28

erlend-aasland added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Aug 30, 2023

erlend-aasland linked an issue Aug 30, 2023 that may be closed by this pull request

sqlite3.iterdump() incompatible with binary data #108590

Closed

2 tasks done

erlend-aasland merged commit 400a1ce into python:main Aug 30, 2023

This comment was marked as outdated.

Sign in to view

bedevere-bot removed the awaiting merge label Aug 30, 2023

miss-islington assigned erlend-aasland Aug 30, 2023

CorvinM mentioned this pull request Aug 30, 2023

[3.12] gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (GH-108657) #108673

Closed

bedevere-bot removed the needs backport to 3.12 only security fixes label Aug 30, 2023

CorvinM mentioned this pull request Aug 30, 2023

[3.11] gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (GH-108657) #108674

Merged

bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 30, 2023

erlend-aasland pushed a commit that referenced this pull request Aug 30, 2023

[3.11] gh-108590 : Fix sqlite3.iterdump for invalid Unicode in TEXT co…

5a6d123

…lumns (GH-108657) (#108674) (cherry picked from commit 400a1ce)

erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023

pythongh-108590 : Revert pythongh-108657 (commit 400a1ce )

cd05820

Reverted per Serhiy's request.

erlend-aasland added a commit that referenced this pull request Aug 30, 2023

gh-108590 : Revert gh-108657 (commit 400a1ce ) (#108686 )

2a3926f

Reverted per Serhiy's request.

erlend-aasland added a commit to erlend-aasland/cpython that referenced this pull request Aug 30, 2023

[3.11] pythongh-108590 : Revert pythongh-108657 (commit 400a1ce ) (pyth…

8888729

…on#108686) (cherry picked from commit 2a3926f) Reverted per Serhiy's request.

erlend-aasland added a commit that referenced this pull request Aug 30, 2023

[3.11] gh-108590 : Revert gh-108657 (commit 400a1ce ) (#108686 ) (#108694 )

6f24420

(cherry picked from commit 2a3926f) Reverted per Serhiy's request.

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. by CorvinM · Pull Request #108657 · python/cpython · GitHub

Repository navigation

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. - #108657

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns.#108657
erlend-aasland merged 7 commits into
python:mainfrom
CorvinM:gh-108590

CorvinM commented Aug 29, 2023 •
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erlend-aasland left a comment

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

This comment was marked as outdated.

miss-islington commented Aug 30, 2023

Uh oh!

miss-islington commented Aug 30, 2023

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. by CorvinM · Pull Request #108657 · python/cpython · GitHub

Repository navigation

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns. - #108657

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns.#108657erlend-aasland merged 7 commits intopython:mainpython/cpython:mainfrom CorvinM:gh-108590CorvinM/cpython:gh-108590Copy head branch name to clipboard

Conversation

CorvinM commented Aug 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erlend-aasland left a comment

Choose a reason for hiding this comment

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

This comment was marked as outdated.

miss-islington commented Aug 30, 2023

Uh oh!

miss-islington commented Aug 30, 2023

Uh oh!

erlend-aasland commented Aug 30, 2023

Uh oh!

CorvinM commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

bedevere-bot commented Aug 30, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gh-108590: Fix sqlite3.iterdump for invalid unicode in text columns.#108657
erlend-aasland merged 7 commits into
python:mainfrom
CorvinM:gh-108590

CorvinM commented Aug 29, 2023 •
edited
Loading