Upgrade libuv to v1.48.0 by niklasr22 · Pull Request #600 · MagicStack/uvloop · GitHub

Upgrades libuv to v1.48.0 which fixes a security vulnerability.

I removed two DNS test cases because they raise an error intended by libuv.

I think the pipeline should be able to pass as it did in the PR in my fork repo. Could someone please trigger a retry?

yeah that error in CI looks like an old flake

I did some investigating about that venerability. I checked if I could reproduce the ‘truncate after 256 bytes’ venerability. I cannot exploit it if I pass hostname as a string, due to this idna encoding line; uvloop accidentally protects you from the libuv venerability:

>>> payload = f'0x{"0"*246}7f000001.example.com'
>>> payload
'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'

>>> import uvloop, asyncio
>>> async def loop_getaddrinfo(addr, port): return await asyncio.get_running_loop().getaddrinfo(addr, port)
>>> uvloop.run(loop_getaddrinfo(payload, 80))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "<stdin>", line 1, in loop_get_addrinfo
  File "uvloop/loop.pyx", line 1528, in getaddrinfo
  File "uvloop/loop.pyx", line 905, in uvloop.loop.Loop._getaddrinfo
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

This is a similar error that socket gives:

>>> import socket
>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label too long)

However, if I pass the hostname as bytes, I can bypass the accidental uvloop protection and exploit libuv:

>>> payload = f'0x{"0"*246}7f000001.example.com'.encode()
>>> payload
b'0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f000001.example.com'
>>> uvloop.run(loop_getaddrinfo(payload, 80))
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

socket, however, isn’t fooled:

>>> socket.getaddrinfo(payload, "80")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mfulmer/miniconda3/envs/lm2/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

I didn’t know about this "0x7f000001" form of hostnames, but, apparently it’s a thing:

>>> socket.getaddrinfo("0x7f000001", "80")
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 80)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 80))]

I can’t find any documentation about why that’s considered a valid hostname. It’s obviously a hex encoding of a 4-byte ipv4 address, but, I’ve never seen it written that way

Anyway, maybe you can turn my investigation into a unit test for the security venerability

regarding the idna encoding error, there's some discussion of whether that error should be handled a different way in the python standard library or not. Just for reference: python/cpython#77139

This reverts commit 281dc2c.

It seems getaddrinfo('', ...) on macOS is equivalent to nodename='localhost'. This is inconsistent with libuv 1.48 which treats empty nodename as EINVAL.

Thanks to @tapple-cisco for the repro

What input would trigger a UnicodeError?

This is what @tapple-cisco mentioned with the vulnerability repro, as well as the BPO. A short example with CPython is like:

>>> payload = f'0x{"0"*246}7f000001.example.com'
>>> import socket; socket.getaddrinfo(payload, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/socket.py", line 964, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/encodings/idna.py", line 173, in encode
    raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long
encoding with 'idna' codec failed

Is encoding with 'idna' codec failed a context exception?

i.e. .__context__.

yeah it's a weird output, let me see

It's __notes__, this particular note is added in 3.12 (changed from a __context__).

Changes ======= * Upgrade libuv to v1.48.0 (#600) (by @niklasr22 @fantix in 7777852 for #596 #615) Fixes ===== * Fix test_create_server_4 with Python 3.12.5 (#614) (by @shadchin in 62f9239) * Use len(os.sched_getaffinity(0)) instead of os.cpu_count() (#591) (by @avkarenow in c8531c2 for #591) * Inline _Py_RestoreSignals() from CPython (#604) (by @befeleme in 8511ba1 for #603)

niklasr22 added 2 commits March 12, 2024 11:11

Upgrade libuv version to 1.48.0

fc3beb5

Remove test cases

281dc2c

This was referenced Aug 14, 2024

Bump libuv to 1.48 #616

Closed

vulnerability in libuv v1.46.0 #615

Closed

fantix reviewed Aug 14, 2024

View reviewed changes

Comment thread tests/test_dns.py

fantix added 5 commits August 13, 2024 23:49

Revert "Remove test cases"

7764b3f

This reverts commit 281dc2c.

Merge remote-tracking branch 'origin/master' into upgrade-libuv

897fee7

Fix for libuv 1.48

f3446e4

Fix for macOS

bbe09cc

It seems getaddrinfo('', ...) on macOS is equivalent to nodename='localhost'. This is inconsistent with libuv 1.48 which treats empty nodename as EINVAL.

Add test

7768f08

Thanks to @tapple-cisco for the repro

fantix requested review from 1st1 and elprans August 14, 2024 04:46

1st1 reviewed Aug 14, 2024

View reviewed changes

Comment thread uvloop/dns.pyx

1st1 reviewed Aug 14, 2024

View reviewed changes

Comment thread uvloop/dns.pyx Outdated

1st1 reviewed Aug 14, 2024

View reviewed changes

crf: add comments and handle possible errors in error handling

c9ff4e4

1st1 approved these changes Aug 14, 2024

View reviewed changes

fantix merged commit 7777852 into MagicStack:master Aug 15, 2024

fantix mentioned this pull request Aug 15, 2024

libuv 1.48.0: getaddrinfo('', 0): OSError: [Errno 22] Invalid argument #596

Closed

fantix mentioned this pull request Aug 15, 2024

uvloop 0.20.0 #617

Merged

                           a2 = self.loop.run_until_complete(
                               self.loop.getaddrinfo(*args, **kwargs))
-                      except socket.gaierror as ex:
+                      except (socket.gaierror, UnicodeError) as ex:

Upgrade libuv to v1.48.0 by niklasr22 · Pull Request #600 · MagicStack/uvloop · GitHub

Uh oh!

Upgrade libuv to v1.48.0 - #600

Upgrade libuv to v1.48.0#600fantix merged 8 commits intoMagicStack:masterMagicStack/uvloop:masterfrom niklasr22:upgrade-libuvniklasr22/uvloop:upgrade-libuvCopy head branch name to clipboard

Conversation

niklasr22 commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

niklasr22 commented Mar 12, 2024

Uh oh!

fantix commented Mar 13, 2024

Uh oh!

tapple-cisco commented Jun 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tapple-cisco commented Jun 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1st1 Aug 14, 2024

Choose a reason for hiding this comment

Uh oh!

fantix Aug 14, 2024

Choose a reason for hiding this comment

Uh oh!

1st1 Aug 14, 2024

Choose a reason for hiding this comment

Uh oh!

1st1 Aug 14, 2024

Choose a reason for hiding this comment

Uh oh!

fantix Aug 14, 2024

Choose a reason for hiding this comment

Uh oh!

fantix Aug 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Upgrade libuv to v1.48.0#600
fantix merged 8 commits into
MagicStack:masterfrom
niklasr22:upgrade-libuv

niklasr22 commented Mar 12, 2024 •
edited
Loading

tapple-cisco commented Jun 21, 2024 •
edited
Loading

tapple-cisco commented Jun 24, 2024 •
edited
Loading

fantix Aug 15, 2024 •
edited
Loading