`_remote_debugging`: unbounded memory growth when recording binary and the thread is sleeping · Issue #151378 · python/cpython · GitHub

Bug report

Bug description:

When recording binary files, pending RLEs are flushed when the stack is changed:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 1014 to 1018 in 29a920e

    
           /* Stack changed - flush any pending RLE first */ 
        
           if (entry->has_pending_rle) { 
        
               if (flush_pending_rle(writer, entry) < 0) { 
        
                   return -1; 
        
               }

or on finalize:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 1077 to 1083 in 29a920e

    
           for (size_t i = 0; i < writer->thread_count; i++) { 
        
               if (writer->thread_entries[i].has_pending_rle) { 
        
                   if (flush_pending_rle(writer, &writer->thread_entries[i]) < 0) { 
        
                       return -1; 
        
                   } 
        
               } 
        
           }

So, a sleeper can cause a memory leak:

cpython/Modules/_remote_debugging/binary_io_writer.c

Lines 996 to 1012 in 29a920e

    
               if (encoding == STACK_REPEAT) { 
        
                   /* Buffer this sample for RLE. 
        
                    * 
        
                    * STACK_REPEAT also covers the "first sample for a fresh thread, 
        
                    * empty stack" case: a new ThreadEntry has prev_stack_depth == 0 
        
                    * and a zero-initialized prev_stack, so compare_stacks() returns 
        
                    * STACK_REPEAT against an empty curr_stack (depth 0). Buffering 
        
                    * it here is correct; the RLE flush path emits it as a normal 
        
                    * STACK_REPEAT record. */ 
        
                   if (GROW_ARRAY(entry->pending_rle, entry->pending_rle_count, 
        
                                  entry->pending_rle_capacity, PendingRLESample) < 0) { 
        
                       return -1; 
        
                   } 
        
                   entry->pending_rle[entry->pending_rle_count].timestamp_delta = delta; 
        
                   entry->pending_rle[entry->pending_rle_count].status = status; 
        
                   entry->pending_rle_count++; 
        
                   entry->has_pending_rle = 1;

Reproduction

Another one found under "let's keep recording binary for a while and see what happens", but the very minimal repro is:

2026-06-11T20:13:39.406750000+0200 maurycy@gimel /Users/maurycy/work/cpython (main d2e27ac?) % ./python.exe -c "
import time
time.sleep(3600)
" & TARGET=$!

sudo ./python.exe -m profiling.sampling attach --binary -r 1000khz -d 300 -o /tmp/rle.bin $TARGET &
sleep 2; PROF=$(pgrep -fn "profiling.sampling attach")
for i in $(seq 5); do printf "t=%2dmin  RSS=%d MB\n" $i $(($(ps -o rss= -p $PROF|tr -d ' ')/1024)); sleep 60; done
[1] 62736
[2] 62737
t= 1min  RSS=46 MB
t= 2min  RSS=348 MB
t= 3min  RSS=645 MB
t= 4min  RSS=945 MB
t= 5min  RSS=1246 MB
Captured 98,200,648 samples in 300.00 seconds
Sample rate: 327,335.49 samples/sec
Error rate: 0.00
  Binary Encoding:
    Records:          1
      RLE repeat:     0 (0.0%) [0 samples]
      Full stack:     1 (100.0%)
      Suffix match:   0 (0.0%)
      Pop-push:       0 (0.0%)
  Frame Efficiency:
    Frames written:   1
    Frames saved:     0 (0.0%)
    Bytes (pre-zstd): 18 B
Warning: missed 201799353 samples from the expected total of 300000001 (67.27%)
Binary profile written to /tmp/rle.bin (98200648 samples)
[2]  + done       sudo ./python.exe -m profiling.sampling attach --binary -r 1000khz -d 300 -o 
2026-06-11T20:18:42.018130000+0200 maurycy@gimel /Users/maurycy/work/cpython (main d2e27ac?) %

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

cc @pablogsal I will take this one, it's making my life under mach_vm_remap testing hard :-)

I believe the proper fix is getting rid of dynamic array

gh-151378: Fix unbounded memory growth when recording binary and the …

[3.15] gh-151378: Fix unbounded memory growth when recording binary a…

Drop the now-redundant emit_batch trim and fix the review nits

gh-152721: Fix quadratic RLE replay time in the profiling binary read…

`_remote_debugging`: unbounded memory growth when recording binary and the thread is sleeping · Issue #151378 · python/cpython · GitHub

Repository navigation

_remote_debugging: unbounded memory growth when recording binary and the thread is sleeping #151378

Bug report

Bug description:

Reproduction

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	/* Stack changed - flush any pending RLE first */
	if (entry->has_pending_rle) {
	if (flush_pending_rle(writer, entry) < 0) {
	return -1;
	}

	for (size_t i = 0; i < writer->thread_count; i++) {
	if (writer->thread_entries[i].has_pending_rle) {
	if (flush_pending_rle(writer, &writer->thread_entries[i]) < 0) {
	return -1;
	}
	}
	}

	if (encoding == STACK_REPEAT) {
	/* Buffer this sample for RLE.
	*
	* STACK_REPEAT also covers the "first sample for a fresh thread,
	* empty stack" case: a new ThreadEntry has prev_stack_depth == 0
	* and a zero-initialized prev_stack, so compare_stacks() returns
	* STACK_REPEAT against an empty curr_stack (depth 0). Buffering
	* it here is correct; the RLE flush path emits it as a normal
	* STACK_REPEAT record. */
	if (GROW_ARRAY(entry->pending_rle, entry->pending_rle_count,
	entry->pending_rle_capacity, PendingRLESample) < 0) {
	return -1;
	}
	entry->pending_rle[entry->pending_rle_count].timestamp_delta = delta;
	entry->pending_rle[entry->pending_rle_count].status = status;
	entry->pending_rle_count++;
	entry->has_pending_rle = 1;

`_remote_debugging`: unbounded memory growth when recording binary and the thread is sleeping · Issue #151378 · python/cpython · GitHub

Repository navigation

_remote_debugging: unbounded memory growth when recording binary and the thread is sleeping #151378

Description

Bug report

Bug description:

Reproduction

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

maurycy commented on Jun 11, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions