Repository navigation
gh-135371: Fix asyncio introspection output to include internal coroutine chains - #135436
Merged
pablogsal merged 8 commits intoJun 14, 2025
Merged
pablogsal merged 8 commits into
pablogsal merged 8 commits into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
…acks This commit refactors the C module to provide richer debugging information by introducing structured data types and exposing internal coroutine call stacks. The improvement transforms the debugging output from showing only external task dependencies to revealing the complete execution context. The key enhancement is the ability to display both the internal coroutine call stack (what the task is doing internally) and the external awaiter chain (what the task is waiting for). This dual perspective makes it much easier to understand complex async execution patterns. Before this change, the debugging output only showed file paths without function names or internal call stacks. With this improvement, developers can now see the complete picture: "foo2 -> sleep" shows the task is in a sleep call within the foo2 function, while the awaiter chain shows the external dependency structure. The implementation replaces raw tuples with structured sequences: - FrameInfo: Function name, filename, and line number - CoroInfo: Call stack and associated task name - TaskInfo: Task ID, name, coroutine stack, and awaiter relationships - ThreadInfo: Thread ID and frame information - AwaitedInfo: Thread ID and list of awaited tasks This structured approach eliminates magic indices and provides better type safety and introspection capabilities.
pablogsal
requested review from
1st1,
ambv,
asvetlov,
kumaraditya303 and
willingc
as code owners
June 12, 2025 17:14
This commit updates the asyncio debugging tools to work with the enhanced structured data from the remote debugging module. The tools now process and display both internal coroutine stacks and external awaiter chains, providing much more comprehensive debugging information. The key improvements include: 1. Enhanced table display: Now shows both "coroutine stack" and "awaiter chain" columns, clearly separating what a task is doing internally vs what it's waiting for externally. 2. Improved tree rendering: Displays complete coroutine call stacks for leaf tasks, making it easier to understand the actual execution state of suspended coroutines. 3. Better cycle detection: Optimized DFS algorithm for detecting await cycles in the task dependency graph. 4. Structured data handling: Updated to work with the new FrameInfo, CoroInfo, TaskInfo, and AwaitedInfo structured types instead of raw tuples. The enhanced output transforms debugging from showing only file paths to revealing function names and complete call stacks, making it much easier to understand complex async execution patterns and diagnose issues in production asyncio applications.
This commit updates the test suite to work with the new structured data types introduced in the remote debugging module and asyncio tools. The tests now use proper FrameInfo, CoroInfo, TaskInfo, and AwaitedInfo structures instead of raw tuples. The test updates ensure that the enhanced debugging capabilities are properly validated, including: 1. Structured data type handling in test mock data 2. Verification of coroutine stack extraction and formatting 3. Testing of both internal coroutine stacks and external awaiter chains 4. Validation of the improved table and tree output formats These changes maintain comprehensive test coverage while adapting to the more sophisticated data structures that enable better async debugging output. The tests verify that the debugging tools correctly process and display the enhanced coroutine execution information, ensuring the new features work reliably in production environments.
This commit updates the What's New documentation to showcase the improved async debugging capabilities introduced in the remote debugging module and asyncio tools. The documentation now demonstrates the enhanced output format that provides much more detailed execution information. The key documentation updates include: 1. Updated example output showing both "coroutine stack" and "awaiter chain" columns in the table format, clearly distinguishing between internal task execution state and external dependencies. 2. Enhanced tree output examples that include function names and complete file paths with line numbers, making the debugging information much more actionable for developers. 3. Addition of cycle detection error example showing how the tools handle problematic await patterns that could indicate programming issues. The updated examples demonstrate how the enhanced debugging output transforms from showing basic file paths to revealing detailed function call stacks and execution context. This improvement makes it significantly easier for developers to understand complex async execution patterns and debug issues in production asyncio applications, especially when dealing with nested coroutines and complex task hierarchies.
pablogsal
force-pushed
the
better_coros-2
branch
from
d1a5d17 to
ec809b5
Compare
June 12, 2025 17:22
Copy link
Copy Markdown
|
Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14. |
Sorry, something went wrong.
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Jun 14, 2025
… coroutine chains (pythonGH-135436) (cherry picked from commit 028309f) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Copy link
Copy Markdown
|
GH-135509 is a backport of this pull request to the 3.14 branch. |
Sorry, something went wrong.
Copy link
Copy Markdown
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️Hi! The buildbot ARM64 Raspbian 3.x (tier-3) has failed when building commit 028309f. What do you need to do:
You can take a look at the buildbot page here: https://buildbot.python.org/#/builders/1678/builds/294 Failed tests:
Failed subtests:
Summary of the results of the build (if available): == Click to see traceback logsTraceback (most recent call last):
File "/home/stan/buildarea/3.x.stan-raspbian.nondebug/build/Lib/test/test_external_inspection.py", line 640, in test_async_global_awaited_by
all_awaited_by = get_all_awaited_by(p.pid)
File "/home/stan/buildarea/3.x.stan-raspbian.nondebug/build/Lib/test/test_external_inspection.py", line 56, in get_all_awaited_by
return unwinder.get_all_awaited_by()
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: Failed to append awaited_by for thread in get_all_awaited_by
|
Sorry, something went wrong.
Copy link
Copy Markdown
⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️Hi! The buildbot AMD64 CentOS9 NoGIL Refleaks 3.x (tier-1) has failed when building commit 028309f. What do you need to do:
You can take a look at the buildbot page here: https://buildbot.python.org/#/builders/1610/builds/1577 Failed tests:
Summary of the results of the build (if available): == Click to see traceback logsremote: Enumerating objects: 19, done.
remote: Counting objects: 5% (1/18)
remote: Counting objects: 11% (2/18)
remote: Counting objects: 16% (3/18)
remote: Counting objects: 22% (4/18)
remote: Counting objects: 27% (5/18)
remote: Counting objects: 33% (6/18)
remote: Counting objects: 38% (7/18)
remote: Counting objects: 44% (8/18)
remote: Counting objects: 50% (9/18)
remote: Counting objects: 55% (10/18)
remote: Counting objects: 61% (11/18)
remote: Counting objects: 66% (12/18)
remote: Counting objects: 72% (13/18)
remote: Counting objects: 77% (14/18)
remote: Counting objects: 83% (15/18)
remote: Counting objects: 88% (16/18)
remote: Counting objects: 94% (17/18)
remote: Counting objects: 100% (18/18)
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 7% (1/14)
remote: Compressing objects: 14% (2/14)
remote: Compressing objects: 21% (3/14)
remote: Compressing objects: 28% (4/14)
remote: Compressing objects: 35% (5/14)
remote: Compressing objects: 42% (6/14)
remote: Compressing objects: 50% (7/14)
remote: Compressing objects: 57% (8/14)
remote: Compressing objects: 64% (9/14)
remote: Compressing objects: 71% (10/14)
remote: Compressing objects: 78% (11/14)
remote: Compressing objects: 85% (12/14)
remote: Compressing objects: 92% (13/14)
remote: Compressing objects: 100% (14/14)
remote: Compressing objects: 100% (14/14), done.
remote: Total 19 (delta 4), reused 4 (delta 4), pack-reused 1 (from 1)
From https://github.com/python/cpython
* branch main -> FETCH_HEAD
Note: switching to '028309fb47869b665f55d10e9eabf7952bf7dbd3'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 028309fb478 gh-135371: Fix asyncio introspection output to include internal coroutine chains (#135436)
Switched to and reset branch 'main'
configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)
make: *** [Makefile:2454: buildbottest] Error 2 |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
Author
|
The raspbian buildbot looks legitimate will take a look soon. |
Sorry, something went wrong.
shuimu5418
pushed a commit
to shuimu5418/cpython001
that referenced
this pull request
Jun 16, 2025
… coroutine chains (python#135436)
lkollar
pushed a commit
to lkollar/cpython
that referenced
this pull request
Jun 19, 2025
… coroutine chains (python#135436)
Pranjal095
pushed a commit
to Pranjal095/cpython
that referenced
this pull request
Jul 12, 2025
… coroutine chains (python#135436)
taegyunkim
pushed a commit
to taegyunkim/cpython
that referenced
this pull request
Aug 4, 2025
… coroutine chains (python#135436)
Agent-Hellboy
pushed a commit
to Agent-Hellboy/cpython
that referenced
this pull request
Aug 19, 2025
… coroutine chains (python#135436)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📚 Documentation preview 📚: https://cpython-previews--135436.org.readthedocs.build/