gh-151022: Fix remote debugging linetable reads by goutamadwant · Pull Request #151036 · python/cpython · GitHub

Fixes gh-151022.

_remote_debugging reads a code object's line table from the target process
with a fixed 4096 byte limit. Valid code objects can exceed that. For example,
Lib/html/entities.py currently compiles to a line table of 37445 bytes, so
RemoteUnwinder.get_stack_trace() fails with RuntimeError: Invalid bytes length.

This keeps the generic remote bytes reader bounded, but gives code object line
tables their own larger limit before parsing. The parser already bounds the
number of line table entries it will scan.

Added a regression test that builds a code object with a line table larger than
4096 bytes and verifies that a same-process stack trace succeeds.

Tests:

./python.exe -m unittest -v test.test_external_inspection.TestSelfStackTrace.test_self_trace_with_large_linetable
./python.exe -m test test_external_inspection -v
make patchcheck

All commit authors signed the Contributor License Agreement.

@maurycy can you review this? :)

@goutamadwant Woah! Thank you for jumping so quickly into this, and sorry for me jumping so slowly. :-) I've left some comments, with a small question mark to @pablogsal himself

@goutamadwant I think there's a missing @requires_remote_subprocess_debugging() decorator :-)

Fixed @maurycy.. added the missing @requires_remote_subprocess_debugging() decorator to the new test class.

@goutamadwant It would be interesting to ensure that the returned linetable is also correct, not -1 :-)

@maurycy Updated this too. The test now checks the returned frame for large_linetable.py and asserts the resolved line number matches the generated RemoteUnwinder call line, so it should catch the -1 fallback case.

@goutamadwant Why exactly 1024 * 1024? :-)

I know that's an arbitrary number but my hunch would be to try to check what sizes are we seeing in the wild, or at least in the stdlib. There's linetable_dist.py script in the original issue that should be helpful after a small modification!

You are right @maurycy :) .. 1024 * 1024 was a conservative cap but not really data driven.

I ran a modified linetable scan against Lib. On current main I see max linetable size 37416 bytes, and the reported copyparty case was also lower, around 19536 bytes.

So I think 64 * 1024 may be a better cap for this PR. It covers the stdlib cases and the reported third party case, while still keeping the remote bytes read bounded much tighter than 1 MiB.

Does that sound reasonable to you ? I can make a change accordingly.

64 KiB sounds reasonable to me given the stdlib and reported third-party numbers.

@goutamadwant

I did not check it but my guess would be that there's much more head room in the MAX_LINETABLE_SIZE now, than in MAX_LINETABLE_ENTRIES. If I'm reading the code right, it fallbacks to -1:

cpython/Modules/_remote_debugging/code_objects.c

Lines 495 to 500 in 29a920e

    
           if (!ok) { 
        
               info.lineno = -1; 
        
               info.end_lineno = -1; 
        
               info.column = -1; 
        
               info.end_column = -1; 
        
           }

@pablogsal

To be honest, I'd assert (proportionally) entries > size or get rid of MAX_LINETABLE_ENTRIES and entry_count. Either one depends on another, or one has to go...

Another solution would be along the lines of code_units (in CachedCodeMetadata?) but I think it's beyond the scope, and we still have to maintain some cap

@maurycy I agree this needs a bit more thought.

From the same scan, the largest stdlib linetable I saw was Lib/html/entities.py with 37416 bytes and about 7465 parsed entries. So for the reported cases, the byte cap is the immediate problem. MAX_LINETABLE_ENTRIES still has enough headroom there.

my thinking is to keep the entry cap as a guard for malformed or pathological linetables and keep this PR focused on the too small byte read limit... I also strengthened the test so it checks a real line number instead of allowing -1.

Changing parse failure from unknown line to an exception feels like a separate behavior change to me. Happy to adjust if you and @pablogsal prefer that in this PR :)

Agreed. With the 64 KiB cap, the separate entry limit is redundant because every iteration consumes at least one byte. Removing it is the cleanest option.

I would keep this PR focused on the read limit. We should open a follow-up to preserve the frame with an unknown location when line metadata cannot be read.

…table

LGTM

Let's land this focused fix and open a follow-up for graceful degradation. If line metadata exceeds the limit or cannot be read, we should preserve the frame with an unknown location instead of dropping the stack.

Thanks a lot for the fix!

Thanks @goutamadwant for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

GH-154187 is a backport of this pull request to the 3.15 branch.

…154187)

goutamadwant requested a review from pablogsal as a code owner June 6, 2026 22:13

bedevere-app Bot added the awaiting review label Jun 6, 2026

bedevere-app Bot mentioned this pull request Jun 6, 2026

_remote_debugging: raising Invalid bytes length on codeobjects with linetable > 4096 #151022

Closed

pythongh-151022 : Fix remote debugging linetable reads

9b0f3ba

goutamadwant force-pushed the gh-151022-remote-debugging-linetable branch from 34645da to 9b0f3ba Compare June 6, 2026 22:19

maurycy reviewed Jun 9, 2026

View reviewed changes

goutamadwant and others added 3 commits June 10, 2026 21:47

Merge branch 'python:main' into pythongh-151022 -remote-debugging-line…

2b79b7c

…table

pythongh-151022 : Strengthen large linetable test

464aedb

pythongh-151022 : Tighten linetable parsing bounds

baec3c9

pablogsal approved these changes Jul 19, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Jul 19, 2026

pablogsal merged commit aeedae8 into python:main Jul 19, 2026
58 checks passed

bedevere-app Bot removed the awaiting merge label Jul 19, 2026

pablogsal added awaiting merge needs backport to 3.15 bugs and security fixes labels Jul 19, 2026

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Jul 19, 2026

pablogsal pushed a commit that referenced this pull request Jul 19, 2026

[3.15] gh-151022 : Fix remote debugging linetable reads (GH-151036 ) (#…

b3ee190

…154187)

maurycy mentioned this pull request Jul 19, 2026

_remote_debugging: Degrade gracefully reads instead of failing the whole read completely #154194

Closed

		# ============================================================================


		class TestSelfStackTrace(RemoteInspectionTestBase):

+                      self.assertEqual(
+                          result.returncode, 0,
+                          f"stdout: {result.stdout}\nstderr: {result.stderr}"


		#include "_remote_debugging.h"

		#define MAX_LINETABLE_SIZE (1024 * 1024)

               #include "_remote_debugging.h"
+              #define MAX_LINETABLE_SIZE (1024 * 1024)
+              #define MAX_LINETABLE_ENTRIES 65536

	if (!ok) {
	info.lineno = -1;
	info.end_lineno = -1;
	info.column = -1;
	info.end_column = -1;
	}

gh-151022: Fix remote debugging linetable reads by goutamadwant · Pull Request #151036 · python/cpython · GitHub

Repository navigation

gh-151022: Fix remote debugging linetable reads - #151036

gh-151022: Fix remote debugging linetable reads#151036pablogsal merged 4 commits intopython:mainpython/cpython:mainfrom goutamadwant:gh-151022-remote-debugging-linetablegoutamadwant/cpython:gh-151022-remote-debugging-linetableCopy head branch name to clipboard

Conversation

goutamadwant commented Jun 6, 2026

Uh oh!

python-cla-bot Bot commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pablogsal commented Jun 6, 2026

Uh oh!

maurycy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maurycy Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maurycy Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pablogsal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Jul 19, 2026

Uh oh!

bedevere-app Bot commented Jul 19, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gh-151022: Fix remote debugging linetable reads#151036
pablogsal merged 4 commits into
python:mainfrom
goutamadwant:gh-151022-remote-debugging-linetable

python-cla-bot Bot commented Jun 6, 2026 •
edited
Loading

maurycy Jun 9, 2026 •
edited
Loading

maurycy Jun 9, 2026 •
edited
Loading