Repository navigation
profiling.sampling crash when using --async-aware mode with --binary or --gecko #152434
Description
Bug report
Bug description:
The Gecko and binary collectors don't currently handle AwaitedInfo and crash when used with --async-aware mode.
For --binary we could flatten async samples into frame stacks, but replaying to a collector like --live (and potential future collectors which handle async natively) would lose async details. A better option could be to change to the binary format to store task information.
For --gecko we can just implement the same flattening logic like other collectors have.
Reproducer:
repro_async_issue.py
import asyncio
async def w():
s = 0
for i in range(1_000_000):
s += i
await asyncio.sleep(0)
return s
async def main():
await asyncio.gather(w(), w())
asyncio.run(main())python.exe -m profiling.sampling run --gecko --async-aware ./repro_async_issue.py
Traceback (most recent call last):
File "/Users/lkollar/github/cpython/Lib/runpy.py", line 201, in _run_module_as_main
return _run_code(code, main_globals, None,
"__main__", mod_spec)
File "/Users/lkollar/github/cpython/Lib/runpy.py", line 87, in _run_code
exec(code, run_globals)
~~~~^^^^^^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/__main__.py", line 65, in <module>
main()
~~~~^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 977, in main
_main()
~~~~~^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 1133, in _main
handler(args)
~~~~~~~^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 1280, in _handle_run
collector = sample(
process.pid,
...<9 lines>...
blocking=args.blocking,
)
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 525, in sample
profiler.sample(collector, duration_sec, async_aware=async_aware)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 188, in sample
raise e from None
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 169, in sample
flush_pending()
~~~~~~~~~~~~~^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 148, in flush_pending
collector.collect(prev_stack, timestamps_us=ts)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/gecko_collector.py", line 255, in collect
for thread_info in interpreter_info.threads:
^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: '_remote_debugging.AwaitedInfo' object has no attribute 'threads'. Did you mean '.thread_id' instead of '.threads'?Traceback (most recent call last):
File "/Users/lkollar/github/cpython/Lib/runpy.py", line 201, in _run_module_as_main
return _run_code(code, main_globals, None,
"__main__", mod_spec)
File "/Users/lkollar/github/cpython/Lib/runpy.py", line 87, in _run_code
exec(code, run_globals)
~~~~^^^^^^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/__main__.py", line 65, in <module>
main()
~~~~^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 977, in main
_main()
~~~~~^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 1133, in _main
handler(args)
~~~~~~~^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/cli.py", line 1280, in _handle_run
collector = sample(
process.pid,
...<9 lines>...
blocking=args.blocking,
)
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 525, in sample
profiler.sample(collector, duration_sec, async_aware=async_aware)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 188, in sample
raise e from None
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/sample.py", line 176, in sample
collector.collect(stack_frames)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/Users/lkollar/github/cpython/Lib/profiling/sampling/binary_collector.py", line 84, in collect
self._writer.write_sample(stack_frames, timestamp_us)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object cannot be interpreted as an integerCC: @pablogsal
CPython versions tested on:
3.16
Operating systems tested on:
Linux, macOS
Linked PRs
- gh-152434: Fix async-aware Gecko collection #152442
- gh-152434: Block --async-aware with --binary #152444
- [3.15] gh-152434: Block --async-aware with --binary (GH-152444) #152446
- [3.15] gh-152434: Fix async-aware Gecko collection (GH-152442) #152450
- gh-152434: Avoid Gecko export in async-aware test #152496
- [3.15] gh-152434: Avoid Gecko export in async-aware test (GH-152496) #152498
Activity
gh-152434: Block --async-aware with --binary (#152444)
[3.15] gh-152434: Block --async-aware with --binary (GH-152444) (#152446
gh-152434: Fix async-aware Gecko collection (#152442)
[3.15] gh-152434: Fix async-aware Gecko collection (GH-152442) (#152450)
gh-152434: Avoid Gecko export in async-aware test (#152496)
[3.15] gh-152434: Avoid Gecko export in async-aware test (GH-152496) (#…