Repository navigation
Inconsistent behavior of asyncio.Server.wait_closed in Python 3.12 versus earlier releases #120866
Description
Bug report
Bug description:
The issue fixed in #111424 is only available in Python 3.12. As a result, the example below prematurely closes a connection when using an older Python version. The old behavior may lead to bugs that are hard to catch, and the difference in behavior implies that applications must use workarounds to support multiple Python versions.
If backporting the fix is not an option, it would be helpful to add a note to the documentation of asyncio.Server.wait_closed that it changed in Python 3.12.
Related issues: #104344 (same cause, going from 3.11 to 3.12), #79033
minimal example
server.py
import asyncio
from functools import partial
async def handler(stop_event, reader, writer):
print("Client connected")
msg = (await reader.readline()).decode()
print(f"Received: {msg.strip()}")
await asyncio.sleep(1)
if "STOP" in msg:
print("Client requests stop. Setting stop event")
stop_event.set()
await asyncio.sleep(1)
print("----> Respond with same message <----")
writer.write(msg.encode())
await writer.drain()
writer.close()
await writer.wait_closed()
print("----> Closing connection <----")
async def main():
print("Starting server")
stop_event = asyncio.Event()
socket_path = "socket"
server = await asyncio.start_unix_server(partial(handler, stop_event), socket_path)
print(f"Server listening at: {socket_path}")
async with server:
await stop_event.wait()
print("Server stopping")
print("Server stopped")
if __name__ == "__main__":
asyncio.run(main())client.py
import asyncio
async def talk(msg):
socket_path = "socket"
print(f"Connecting to: {socket_path}")
reader, writer = await asyncio.open_unix_connection(socket_path)
print("Connected")
print(f"Sending message: {msg}")
writer.write(f"{msg}\n".encode())
await writer.drain()
res = (await reader.readline()).decode()
print(f"Received response: {res[:-1]}")
print("Closing connection")
writer.close()
await writer.wait_closed()
async def main():
print("Starting client")
await talk("Hello")
await talk("STOP")
print("Stopping client")
if __name__ == "__main__":
asyncio.run(main())This is a minimal working example distilled from a more complex use case where the stop_event is set in a function that receives the request and creates the response.
One possible workaround is to postpone setting the stop_event to the end of the handler function. That would be easy in this example, but more involved in the case where I ran into this issue.
Python 3.12 and 3.13 output
server.py
Starting server Server listening at: socket Client connected Received: Hello ----> Respond with same message <---- ----> Closing connection <---- Client connected Received: STOP Client requests stop. Setting stop event Server stopping ----> Respond with same message <---- ----> Closing connection <---- Server stopped
client.py
Starting client Connecting to: socket Connected Sending message: Hello Received response: Hello Closing connection Connecting to: socket Connected Sending message: STOP Received response: STOP Closing connection Stopping client
Python 3.9, 3.10 and 3.11 output
Note that the handler does not complete when it receives STOP.
server.py
Starting server Server listening at: socket Client connected Received: Hello ----> Respond with same message <---- ----> Closing connection <---- Client connected Received: STOP Client requests stop. Setting stop event Server stopping Server stopped
client.py
Starting client Connecting to: socket Connected Sending message: Hello Received response: Hello Closing connection Connecting to: socket Connected Sending message: STOP Received response: Closing connection Stopping client
CPython versions tested on:
3.9, 3.10, 3.11, 3.12, 3.13
Operating systems tested on:
Linux
Linked PRs
- gh-120866: Document behavior change of asyncio.Server.wait_closed() since 3.12 #154676
- [3.15] gh-120866: Document behavior change of asyncio.Server.wait_closed() since 3.12 (GH-154676) #154677
- [3.14] gh-120866: Document behavior change of asyncio.Server.wait_closed() since 3.12 (GH-154676) #154678
- [3.13] gh-120866: Document behavior change of asyncio.Server.wait_closed() since 3.12 (GH-154676) #154679
Activity
tovrstra commented on Jun 22, 2024
Here is a workaround that mimics the implementation in Python 3.12.1 when using a Python version without the fix in #111424. One can add the following code right before print("Server stopped") in server.py:
This may be a useful snippet for implementing a server that should work across different Python versions.
Fix SSH server shutdown with active clients
Fix SSH server shutdown with active clients
pythongh-120866: Document behavior change of asyncio.Server.wait_clos…
pythongh-120866: Document behavior change of asyncio.Server.wait_clos…
gh-120866: Document behavior change of asyncio.Server.wait_closed() s…
[3.13] gh-120866: Document behavior change of asyncio.Server.wait_clo…
[3.14] gh-120866: Document behavior change of asyncio.Server.wait_clo…
[3.15] gh-120866: Document behavior change of asyncio.Server.wait_clo…
pythongh-120866: Document behavior change of asyncio.Server.wait_clos…