Repository navigation
Support for weak references in frames #102960
Description
It's occasionally useful to be able to link information to active frames. Two recent examples:
- https://stackoverflow.com/questions/72270154/weakref-weakkeydictionary-to-frame-frametype-objects
- gh-102864: Remove access to f_locals in pdb #102865
Currently, since frames don't support being weakly referenced, custom lifecycle management is required in this case. That's unfortunate because errors in cleaning out the references will lead to memory leaks of dead frames and their respective locals.
The straightforward solution is to add weakref support to frame objects. The performance hit is limited to setting NULL on the f_weakreflist field on frame object creation, and later a NULL check (for whether there are any weakrefs) at the frame object's deallocation. This is what the related PR does.
I ran the PR and the base main branch at the time against pyperformance 1.0.6. The raw results are in benchmarks.zip but the gist is that there are no significant differences between the two branches in any of the benchmarks executed.
Activity
markshannon commented on Mar 23, 2023
Do you have a link to the results in a legible form?
ambv commented on Mar 23, 2023
I'll get that for you tomorrow. In the mean time you can:
ambv commented on Mar 24, 2023
The only "Significant" one is tornado_http and this one claims it's in favor of the weakreflist introduction so clearly noise.
glyph commented on Mar 29, 2023
There's some Deferred debugging stuff that has caused significant angst with hard references to frame objects. Weakrefs here would be hugely useful. I'm not sure exactly how we'd use them (the cycles and attendant leaks were such a problem that we have mostly given up on this sort of thing and I'd need to load the whole feature back into my head) but I definitely want this if it is a possibility!
tacaswell commented on Mar 30, 2023
We have an open issue on Matplotlib due to frame related reference cycles matplotlib/matplotlib#25406
markshannon commented on Mar 30, 2023
This is not the way to fix memory leaks.
If you really must have cycles involving frames, then you can break the cycle like this:
class SupportsWeakRefs: pass
frame -> weakref() -> SupportsWeakRefs() -> frame.
Although I have to wonder why plt.savefig needs to do strange things to the stack.
tacaswell commented on Mar 30, 2023
plt.savefig is how the use found it but the cycles are from calls much deeper in the code (we were caching an Exception object 🐑 , and it looks like some versions of pillow and pyparsing also are doing something with cached exceptions). I do not think there is a compelling reason we (Matplotlib) needs this, but we were bit by frame cycles someplace.
gaogaotiantian commented on Jan 20, 2025
@ambv and @markshannon , do we have plans to move forward on this? pdb kind of needs it. pdb (and bdb) often keeps some data for a specific frame, and it should last during the frame life cycle. It's a bit cumbersome to properly release the reference to the frames - either we hold it for unnecessarily long (and possibly forever), or we need a lot of extra code to release the reference at different places.
gaogaotiantian commented on Feb 16, 2025
ping here again for whether we can have this - doesn't seem like there are a lot of downsides. The perf looks the same and the code change is trivial.
pdb and bdb can use this to keep frame specific data and it can be really helpful.
markshannon commented on Jun 30, 2026
Anyone want to implement this?
@gaogaotiantian?
@ambv?
ambv commented on Jun 30, 2026
I will implement this 👍🏻
gh-102960: Make frames weak-referenceable (#152838)