FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-98433: Fix quadratic time idna decoding. by gpshead · Pull Request #99092 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

gh-98433: Fix quadratic time idna decoding. #99092

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gpshead merged 8 commits into from
Nov 8, 2022
Conversation 19 Commits 8 Checks 0 Files changed
Merged

gh-98433: Fix quadratic time idna decoding. #99092

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
365a6cb
gh-98433: Fix quadratic time idna decoding.
gpshead Nov 4, 2022
4bf248b
drop the timing test, add an upfront limit.
gpshead Nov 4, 2022
db498cb
Expand the label limit comments.
gpshead Nov 4, 2022
be30a48
Also add a short "too long" test assert.
gpshead Nov 4, 2022
038bbcd
Refactor into nicer code. Expand the NEWS entry.
gpshead Nov 4, 2022
cf3a20e
news wording tweak.
gpshead Nov 4, 2022
bd51456
more RFC links and explanation.
gpshead Nov 4, 2022
5a3b037
assertRaisesRegex
gpshead Nov 7, 2022
File filter

Filter by extension

Filter by extension .py  (2) .rst  (1) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
42 changes: 25 additions & 17 deletions Lib/encodings/idna.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -39,23 +39,21 @@ def nameprep(label):

# Check bidi
RandAL = [stringprep.in_table_d1(x) for x in label]
for c in RandAL:
if c:
# There is a RandAL char in the string. Must perform further
# tests:
# 1) The characters in section 5.8 MUST be prohibited.
# This is table C.8, which was already checked
# 2) If a string contains any RandALCat character, the string
# MUST NOT contain any LCat character.
if any(stringprep.in_table_d2(x) for x in label):
raise UnicodeError("Violation of BIDI requirement 2")

# 3) If a string contains any RandALCat character, a
# RandALCat character MUST be the first character of the
# string, and a RandALCat character MUST be the last
# character of the string.
if not RandAL[0] or not RandAL[-1]:
raise UnicodeError("Violation of BIDI requirement 3")
if any(RandAL):
# There is a RandAL char in the string. Must perform further
# tests:
# 1) The characters in section 5.8 MUST be prohibited.
# This is table C.8, which was already checked
# 2) If a string contains any RandALCat character, the string
# MUST NOT contain any LCat character.
if any(stringprep.in_table_d2(x) for x in label):
raise UnicodeError("Violation of BIDI requirement 2")
# 3) If a string contains any RandALCat character, a
# RandALCat character MUST be the first character of the
# string, and a RandALCat character MUST be the last
# character of the string.
if not RandAL[0] or not RandAL[-1]:
raise UnicodeError("Violation of BIDI requirement 3")

return label

Expand Down Expand Up @@ -103,6 +101,16 @@ def ToASCII(label):
raise UnicodeError("label empty or too long")

def ToUnicode(label):
if len(label) > 1024:
# Protection from https://github.com/python/cpython/issues/98433.
# https://datatracker.ietf.org/doc/html/rfc5894#section-6
# doesn't specify a label size limit prior to NAMEPREP. But having
# one makes practical sense.
# This leaves ample room for nameprep() to remove Nothing characters
# per https://www.rfc-editor.org/rfc/rfc3454#section-3.1 while still
# preventing us from wasting time decoding a big thing that'll just
# hit the actual <= 63 length limit in Step 6.
raise UnicodeError("label way too long")
# Step 1: Check for ASCII
if isinstance(label, bytes):
pure_ascii = True
Expand Down
6 changes: 6 additions & 0 deletions Lib/test/test_codecs.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -1552,6 +1552,12 @@ def test_builtin_encode(self):
self.assertEqual("pyth\xf6n.org".encode("idna"), b"xn--pythn-mua.org")
self.assertEqual("pyth\xf6n.org.".encode("idna"), b"xn--pythn-mua.org.")

def test_builtin_decode_length_limit(self):
with self.assertRaisesRegex(UnicodeError, "way too long"):
(b"xn--016c"+b"a"*1100).decode("idna")
with self.assertRaisesRegex(UnicodeError, "too long"):
(b"xn--016c"+b"a"*70).decode("idna")

def test_stream(self):
r = codecs.getreader("idna")(io.BytesIO(b"abc"))
r.read(3)
Expand Down
14 changes: 14 additions & 0 deletions Misc/NEWS.d/next/Security/2022-11-04-09-29-36.gh-issue-98433.l76c5G.rst
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
The IDNA codec decoder used on DNS hostnames by :mod:`socket` or :mod:`asyncio`
related name resolution functions no longer involves a quadratic algorithm.
This prevents a potential CPU denial of service if an out-of-spec excessive
length hostname involving bidirectional characters were decoded. Some protocols
such as :mod:`urllib` http ``3xx`` redirects potentially allow for an attacker
to supply such a name.

Individual labels within an IDNA encoded DNS name will now raise an error early

gpshead Nov 4, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm okay with leaving this length limit out either entirely - or just out of the backports if release managers are uncomfortable with this in a bugfix.

I view it as protection from other as yet unknown future idna or punycode decoder logic bugs to avoid wasting cpu time. The only reference numbers related to my arbitrary input length limit choice are: A DNS hostname is 255 bytes at most. A label within a DNS name is 63 bytes at most.

The "Nothing" characters in question that are removed in nameprep() in https://github.com/python/cpython/blob/main/Lib/encodings/idna.py#L18 are non-ascii unicode oddities.

Other IDNA decoding libraries in the wild like the well known ICU library do not limit the length at this level of API, instead leaving that to higher levels in the application:
during IDNA decoding if they are longer than 1024 unicode characters given that
each decoded DNS label must be 63 or fewer characters and the entire decoded
DNS name is limited to 255. Only an application presenting a hostname or label
consisting primarily of :rfc:`3454` section 3.1 "Nothing" characters to be
removed would run into of this new limit. See also :rfc:`5894` section 6 and
:rfc:`3491`.
Back | FazBrowse Home | New Git URL