gh-87720 Fix additional header refolding-quoting edge case by bitdancer · Pull Request #134271 · python/cpython · GitHub

In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, so the encoding check never happened. To fix this, we look inside the current unit to see if has anything that was originally encoded.

I'm not really happy with the method name 'has_token_type', but I haven't thought of anything I like better. Ideas welcome.

Issue: email: encoded headers lose their quoting when refolded #87720

In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, so the encoding check never happened. To fix this, we look inside the current unit to see if has anything that was originally encoded.

@medmunds Do you feel like reviewing this? :)

I'm not really happy with the method name 'has_token_type', but I haven't thought of anything I like better. Ideas welcome.

contains_token_type? (Though I'm not convinced that's any better; has_token_type makes sense to me too.)

This fix LGTM. But I found another corner case while looking at it. Here's the failing test:

--- a/Lib/test/test_email/test__header_value_parser.py	(revision ebde4e897750963b82bcae785949ac11b3877640)
+++ b/Lib/test/test_email/test__header_value_parser.py	(date 1748289921461)
@@ -3101,6 +3101,8 @@
              '=?utf-8?q?a=2C?=\n'
              ' 123456789012345678901234567890123456\n'
              ' <abc@example.com>\n'),
+            ('abc@example.com (, but definitely not also sent to def@example.net)',
+             'abc@example.com (, but definitely not\n also sent to def@example.net)\n'),
         ]
         for (to, folded) in cases:
             with self.subTest(to=to):

(This case belongs in a new test covering comment folding, but you'll get the point. Guessing I should open a new issue—it's more like the earlier quoted-string problem. I bring it up here in case it might help you spot other xtext variations needing attention.)

(Also, is this on Seth's radar yet?)

This PR is stale because it has been open for 30 days with no activity.

Still looks good.

This probably should get security backports.

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

GH-159065 is a backport of this pull request to the 3.11 branch.

GH-159066 is a backport of this pull request to the 3.12 branch.

GH-159067 is a backport of this pull request to the 3.14 branch.

GH-159068 is a backport of this pull request to the 3.13 branch.

GH-159069 is a backport of this pull request to the 3.15 branch.

…-134271) (#159067) In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, and that higher level unit was not a ptext or vtext, so the encoding check failed. To fix this, we look inside the current unit to see if has anything that was originally encoded. (cherry picked from commit 5a22a62) Co-authored-by: R. David Murray <rdmurray@bitdance.com>

Fixes #87720.

…-134271) (#159069) In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, and that higher level unit was not a ptext or vtext, so the encoding check failed. To fix this, we look inside the current unit to see if has anything that was originally encoded. (cherry picked from commit 5a22a62) Co-authored-by: R. David Murray <rdmurray@bitdance.com>

Thanks @bitdancer for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

GH-160456 is a backport of this pull request to the 3.13 branch.

…-134271) (#159066) gh-87720 Fix additional header refolding-quoting edge case (GH-134271) In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, and that higher level unit was not a ptext or vtext, so the encoding check failed. To fix this, we look inside the current unit to see if has anything that was originally encoded. (cherry picked from commit 5a22a62) Co-authored-by: R. David Murray <rdmurray@bitdance.com>

…-134271) (#160456) gh-87720 Fix additional header refolding-quoting edge case (GH-134271) In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, and that higher level unit was not a ptext or vtext, so the encoding check failed. To fix this, we look inside the current unit to see if has anything that was originally encoded. (cherry picked from commit 5a22a62) Co-authored-by: R. David Murray <rdmurray@bitdance.com> Co-authored-by: T. Wouters <thomas@python.org>

…-134271) (#159065) gh-87720 Fix additional header refolding-quoting edge case (GH-134271) In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, and that higher level unit was not a ptext or vtext, so the encoding check failed. To fix this, we look inside the current unit to see if has anything that was originally encoded. (cherry picked from commit 5a22a62) Co-authored-by: R. David Murray <rdmurray@bitdance.com>

Fix additional refolding-encoding edge case.

ebde4e8

In this case, the higher level syntactic unit fit on the remainder of the line in un-encoded format, so the encoding check never happened. To fix this, we look inside the current unit to see if has anything that was originally encoded.

bitdancer requested a review from a team as a code owner May 19, 2025 19:10

bedevere-app Bot mentioned this pull request May 19, 2025

email: encoded headers lose their quoting when refolded #87720

Closed

bedevere-app Bot added the awaiting core review label May 19, 2025

bitdancer added the skip news label May 19, 2025

medmunds approved these changes May 26, 2025

View reviewed changes

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 24, 2026

bitdancer added 4 commits October 8, 2026 16:12

Merge branch 'master' into another_refolding_edge_case

58fa449

Better name for new function.

d97bd40

Add news item.

8b106ae

Fix markup.

44115bf

medmunds reviewed Oct 8, 2026

View reviewed changes

bitdancer merged commit 5a22a62 into python:main Oct 9, 2026
55 checks passed

bedevere-app Bot removed the awaiting core review label Oct 9, 2026

bitdancer added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes needs backport to 3.13 only security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 9, 2026

bedevere-app Bot removed the needs backport to 3.11 only security fixes label Oct 9, 2026

bedevere-app Bot removed the needs backport to 3.12 only security fixes label Oct 9, 2026

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 9, 2026

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Oct 9, 2026

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 9, 2026

bitdancer deleted the another_refolding_edge_case branch October 9, 2026 20:00

AlexWaygood mentioned this pull request Oct 22, 2026

Fix stdlib stubtest on Python 3.14.9 and 3.15.1 python/typeshed#16596

Merged

hugovk mentioned this pull request Oct 23, 2026

[3.13] gh-87720 Fix additional header refolding-quoting edge case (GH-134271) #159068

Closed

hugovk added the needs backport to 3.13 only security fixes label Oct 23, 2026

bedevere-app Bot removed the needs backport to 3.13 only security fixes label Oct 23, 2026

leandrodamascena mentioned this pull request Oct 25, 2026

email: generating messages got slower after gh-87720, since folding walks the parse tree repeatedly #160675

Open

gh-87720 Fix additional header refolding-quoting edge case by bitdancer · Pull Request #134271 · python/cpython · GitHub

Repository navigation

gh-87720 Fix additional header refolding-quoting edge case - #134271

gh-87720 Fix additional header refolding-quoting edge case#134271bitdancer merged 5 commits intopython:mainpython/cpython:mainfrom bitdancer:another_refolding_edge_casebitdancer/cpython:another_refolding_edge_caseCopy head branch name to clipboard

Conversation

bitdancer commented May 19, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bitdancer commented May 19, 2025

Uh oh!

medmunds commented May 26, 2025

Uh oh!

medmunds left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 24, 2026

Uh oh!

medmunds left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Oct 9, 2026

Uh oh!

miss-islington-app Bot commented Oct 9, 2026

Uh oh!

miss-islington-app Bot commented Oct 9, 2026

Uh oh!

miss-islington-app Bot commented Oct 9, 2026

Uh oh!

miss-islington-app Bot commented Oct 9, 2026

Uh oh!

bedevere-app Bot commented Oct 9, 2026

Uh oh!

bedevere-app Bot commented Oct 9, 2026

Uh oh!

bedevere-app Bot commented Oct 9, 2026

Uh oh!

bedevere-app Bot commented Oct 9, 2026

Uh oh!

bedevere-app Bot commented Oct 9, 2026

Uh oh!

medmunds commented Oct 9, 2026

Uh oh!

miss-islington-app Bot commented Oct 23, 2026

Uh oh!

bedevere-app Bot commented Oct 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-87720 Fix additional header refolding-quoting edge case#134271
bitdancer merged 5 commits into
python:mainfrom
bitdancer:another_refolding_edge_case

bitdancer commented May 19, 2025 •
edited by bedevere-app Bot
Loading