gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter by matthieucan · Pull Request #136877 · python/cpython · GitHub

With those changes, the MIME parameter parser discards parameters with an invalid section number that uses a digit not convertible to integer such as super-script "²" or "𐩃" (Kharosthi number).

For backwards compatibility, keep accepting non-ASCII digits that can be converted to integers, such as NKO digits.

Before:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    reg('Content-Disposition', 'inline; foo*²=bar')
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 604, in __call__
    return self[name](name, value)
           ~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 192, in __new__
    cls.parse(value, kwds)
    ~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 448, in parse
    kwds['parse_tree'] = parse_tree = cls.value_parser(value)
                                      ~~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2736, in parse_content_disposition_header
    disp_header.append(parse_mime_parameters(value[1:]))
                       ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2601, in parse_mime_parameters
    token, value = get_parameter(value)
                   ~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2464, in get_parameter
    token, value = get_section(value)
                   ~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2417, in get_section
    section.number = int(digits)
                     ~~~^^^^^^^^
ValueError: invalid literal for int() with base 10: '²'

After:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
'inline;'

Issue: email: ValueError in get_section when parsing header with non-ASCII digit #87112

Suggested change

      
                while value and '0' <= value[0] <= '9':
          
                while value and ('0' <= value[0] <= '9'):

It will a bit clearer. Or you can still use a separate function to make it even cleareer. The bottleneck won't be the function call IMO.

I did that, but not the separate function. It was my understanding that @StanFromIreland was leaning towards not having an inner function

This is fine, I was against the function to check if it is in a dictionary.

Thank you! Just moved it to a separate function for extra-clarity

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts?
Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts? Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.

Thank you for looking into this.

In my understanding, those are the possible scenarios:

The section number is made of ASCII digits ('0' <= d <= '9')
- Valid, remains unchanged with this PR
The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE)
- This was accepted before, without a defect
- This PR, in its current state, rejects those (they are ignored and trigger a defect)
- ➡️ ❓ @encukou Do I understand correctly that they should be accepted, and not raise a defect (even if they are not RFC-valid)? Or should they be accepted and raise a defect?
The section number is made of digits (isdigit(d)) which cannot be converted to integers (not int(d)), e.g. ²
- This used to fail with a raised exception
- This PR, in its current state, rejects those (they are ignored and trigger a defect)
The section number is not made of digits (not isdigit(d))
- Invalid, remains unchanged with this PR

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.

Thank you, I agree. This is now implemented.

This PR is stale because it has been open for 30 days with no activity.

I think that what we should to do here is change the incorrect isdigit to isdecimal and add as_ew_allowed=False to MIMEVersion. If I'm reading rfc 6532 correctly (though it is quite possible I'm not) then non-ascii decimal digits are not a defect unless you are serializing to ascii. (They should already show up as non-ascii defects when parsing with utf8=False.) If we make those changes that should eliminate the exception, and the new code proposed in #122540 should raise an error if the header is serialized with utf8=False.

This PR is stale because it has been open for 90 days with no activity.

matthieucan added 2 commits July 20, 2025 17:43

Only allow 0-9 digits in MIME parameter section numbers

ff4ca5c

Use gh bug nubmer

656b6fb

matthieucan requested a review from a team as a code owner July 20, 2025 15:59

bedevere-app Bot added the awaiting review label Jul 20, 2025

bedevere-app Bot mentioned this pull request Jul 20, 2025

email: ValueError in get_section when parsing header with non-ASCII digit #87112

Open

Add news entry

75636a5

StanFromIreland reviewed Jul 20, 2025

View reviewed changes

Comment thread Lib/email/_header_value_parser.py Outdated

Comment thread Lib/email/_header_value_parser.py Outdated

Comment thread Lib/email/_header_value_parser.py Outdated

picnixz changed the title gh-87112: Do not fail when non 0-9 digit is used as section number in MIME header parameter gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter Jul 20, 2025

picnixz reviewed Jul 20, 2025

View reviewed changes

Comment thread Lib/test/test_email/test__header_value_parser.py

matthieucan added 2 commits July 20, 2025 19:17

Comments from PR review, remove inner function

b15c404

Rephrase News entry

7cba42f

StanFromIreland reviewed Jul 20, 2025

View reviewed changes

Comment thread Lib/email/_header_value_parser.py Outdated

Comment thread Lib/test/test_email/test__header_value_parser.py Outdated

picnixz reviewed Jul 20, 2025

View reviewed changes

matthieucan added 2 commits July 20, 2025 19:35

Comments from PR review, improve phrasing

d2d59ee

Use separate function

54bbbb7

AA-Turner added the sprint label Jul 20, 2025

matthieucan added 2 commits July 27, 2025 14:16

Accept digits that are convertible to int for backwards compatibility

f12e424

Update news message

b364339

matthieucan changed the title gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter Jul 27, 2025

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 27, 2026

github-project-automation Bot added this to Sprint Apr 27, 2026

github-project-automation Bot moved this to Todo in Sprint Apr 27, 2026

github-actions Bot removed the stale Stale PR or inactive for long period of time. label May 15, 2026

github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 17, 2026

                                                     "found {}".format(value))
                   digits = ''
-                  while value and value[0].isdigit():
+                  while value and '0' <= value[0] <= '9':

	while value and '0' <= value[0] <= '9':
	while value and ('0' <= value[0] <= '9'):

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter by matthieucan · Pull Request #136877 · python/cpython · GitHub

Uh oh!

Uh oh!

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter - #136877

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter#136877matthieucan wants to merge 9 commits intopython:mainpython/cpython:mainfrom matthieucan:fix-issue-87112matthieucan/cpython:fix-issue-87112Copy head branch name to clipboard

Conversation

matthieucan commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

matthieucan Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

StanFromIreland Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

matthieucan Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

encukou commented Jul 22, 2025

Uh oh!

matthieucan commented Jul 22, 2025

Uh oh!

encukou commented Jul 23, 2025

Uh oh!

matthieucan commented Jul 27, 2025

Uh oh!

github-actions Bot commented Apr 27, 2026

Uh oh!

bitdancer commented Apr 27, 2026

Uh oh!

github-actions Bot commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter#136877
matthieucan wants to merge 9 commits into
python:mainfrom
matthieucan:fix-issue-87112

matthieucan commented Jul 20, 2025 •
edited
Loading