IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop · Issue #93896 · python/cpython · GitHub

in https://github.com/python/cpython/pull/31799/files#diff-1f2ae0f6c6010caf9d5f1c80cd6033a796ffe2b60554f5df84f554f4a08e622b the calls to asyncio.set_event_loop() were removed which breaks aiohttp aio-libs/aiohttp#6757 and asyncio.SafeChildWatcher

this looks like an intentional breaking change - and if it is should be documented

import sys
import asyncio


async def run_subprocess():
    proc = await asyncio.create_subprocess_shell(
        "exit 0",
        stdin=asyncio.subprocess.DEVNULL,
        stdout=asyncio.subprocess.DEVNULL,
        stderr=asyncio.subprocess.DEVNULL,
    )
    await proc.wait()
    print("success!")


async def amain():
    await asyncio.to_thread(asyncio.run, run_subprocess())


def main():
    asyncio.get_event_loop_policy().set_child_watcher(asyncio.SafeChildWatcher())
    asyncio.run(amain())


if __name__ == "__main__":
    sys.exit(main())

 graingert@superjacent  ~/projects  python3.10 demo_subprocess.py
success!
 graingert@superjacent  ~/projects  python3.11 demo_subprocess.py 
Traceback (most recent call last):
  File "/home/graingert/projects/demo_subprocess.py", line 26, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 22, in main
    asyncio.run(amain())
    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 181, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 115, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 17, in amain
    await asyncio.to_thread(asyncio.run, run_subprocess())
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 181, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 115, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/graingert/projects/demo_subprocess.py", line 6, in run_subprocess
    proc = await asyncio.create_subprocess_shell(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1647, in subprocess_shell
    transport = await self._make_subprocess_transport(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/unix_events.py", line 204, in _make_subprocess_transport
    raise RuntimeError("asyncio.get_child_watcher() is not activated, "
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: asyncio.get_child_watcher() is not activated, subprocess support is not installed.

I know there were some efforts to deprecate get/set_event_loop_policy and child watchers #82772 so I think those Deprecations should be applied before asyncio.run/IsolatedAsyncioTestCase stops calling asyncio.set_event_loop()

alternatively is there still time to deprecate the child watchers system and the policy system in favor of
asyncio.Runner(loop_factory=asyncio.ProactorEventLoop/asyncio.SelectorEventLoop/uvloop.uvloop.new_event_loop) before the 3.11 release?

that would be deprecating:

asyncio.get_event_loop() # already deprecated unless the loop is running 
asyncio.set_event_loop()  # asyncio.set_event_loop(None) should probably be exempt
asyncio.get_event_loop_policy()
asyncio.set_event_loop_policy()  # asyncio.set_event_loop_policy(None) should probably be exempt
asyncio.set_child_watcher()

asyncio.get_child_watcher() will remain and will always be asyncio.PidfdChildWatcher on old kernels and asyncio.ThreadedChildWatcher on very old kernels

asyncio.new_event_loop() will issue a DeprecationWarning if the current policy is not the default policy, and then become an alias of

if sys.platform == "win32":
    new_event_loop = ProactorEventLoop
else:
    new_event_loop = SelectorEventLoop

this is also broken with the PidfdChildWatcher

Marking as potential release blocker.

Marking as potential release blocker.

As this is a release blocker can you add @pablogsal as a reviewer for me?

Please, add me as a reviewer to any PR that needs to be merged to address these issues.

https://mail.python.org/archives/list/python-dev@python.org/thread/ORCYBRP432J36LXP32IDX6KLRE7Z646V/

Go ahead with that PR, Kumar.

I created #94593 which fixes this by reverting to the 3.10 and earlier behavior of setting the event loop if no loop factory was supplied. No new parameter is added so can be backport to 3.11.

We should have a separate discussion about the future of the policy system.

see #94597

GH-93896: AAlways set event loop in asyncio.run and IsolatedAsyncioTe…

pythonGH-93896: AAlways set event loop in asyncio.run and IsolatedAsy…

GH-93896: AAlways set event loop in asyncio.run and IsolatedAsyncioTe…

This is fixed in 3.12 (main) and 3.11 by #94593 #94608 respectively.

Thanks everyone!

@ixenion Please be civil. This is not the place to get help, but you can find help for Python issues on discuss.python.org.

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop · Issue #93896 · python/cpython · GitHub

Repository navigation

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop #93896

9 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop · Issue #93896 · python/cpython · GitHub

Repository navigation

IsolatedAsyncioTestCase and asyncio.run no-longer call asyncio.set_event_loop #93896

Description

Activity

graingert commented on Jun 16, 2022

graingert commented on Jun 16, 2022

graingert commented on Jun 17, 2022

graingert commented on Jun 23, 2022

encukou commented on Jun 27, 2022

graingert commented on Jun 30, 2022

9 remaining items

gvanrossum commented on Jul 6, 2022

kumaraditya303 commented on Jul 6, 2022

graingert commented on Jul 6, 2022

kumaraditya303 commented on Jul 6, 2022

ixenion commented on Apr 24, 2024

gvanrossum commented on Apr 24, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions