| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
…e attributes Fixes python#21716. When mypyc compiles a subclass of a builtin type (`Exception` or `dict`) with instance attributes on Python <3.12, it appends a __dict__ slot at tp_dictoffset = sizeof(builtin_base) to hold them. But it emits no subtype-specific tp_dealloc, so the type inherits the base's dealloc which knows nothing about the extra slot. That dict (and everything it references) leaks on every instance destruction. Hit downstream in sqlglot[c] (tobymao/sqlglot#7853) where every raised-and-caught ParseError leaked ~1 KB. On 3.12+ mypyc already avoids adding the extra offset, so only <3.12 is affected. Changes in `mypyc/codegen/emitclass.py`: * Extend the gate that emits tp_dealloc/tp_clear/tp_traverse (and Py_TPFLAGS_HAVE_GC) to fire for any builtin_base subclass with has_dict under capi < 3.12. The existing builtin-base dealloc dance handles the flow. * Set Py_TPFLAGS_HAVE_GC explicitly. PyType_Ready only inherits HAVE_GC from the base when it also inherits both tp_traverse and tp_dealloc; once we supply our own, that inheritance is lost. * Fix a latent offset bug in generate_traverse_for_class / generate_clear_for_class: they used sizeof(struct_name) for the extra dict slot, but for builtin_base classes tp_dictoffset is sizeof(builtin_base). Previously unreachable because the emission gate never fired for builtin_base classes. Changes in `mypyc/codegen/emit.py` (`emit_base_tp_function_call`): * Walk the tp_base chain past any Py_TPFLAGS_HEAPTYPE ancestors before delegating tp_dealloc/tp_traverse/tp_clear. Required for Exception subclasses that inherit through an interpreted Python heap type (e.g. `argparse.ArgumentTypeError` in mypy's own VersionTypeError): calling the heap type's tp_dealloc dispatches through CPython's subtype_dealloc, which reads Py_TYPE(self) (still our subtype), finds our own tp_dealloc, and calls it back — infinite recursion crashing the compiled binary on 3.10/3.11. Our subtype_clear already walks the full MRO and clears all ancestors' fields, so calling the static ancestor's tp_dealloc directly is safe and correct. Regression tests: * testSubclassExceptionNoAttributeLeak (new): a Payload class with a __del__ counter detects the leak deterministically — 1000 fresh exceptions each hold a fresh payload; without the fix all 1000 payloads survive. * testDelForDictSubclass (updated): the stale `sys.version_info >= (3, 12)` guard is dropped since dict subclass __del__ now works correctly on all Python versions.
| # Walk past intermediate heap types (Python or mypyc classes) to reach a | ||
| # static C-level ancestor. Calling a heap type's tp_dealloc/tp_traverse/ | ||
| # tp_clear would dispatch through subtype_dealloc, which uses Py_TYPE(self) | ||
| # (still our subtype) and re-enters our own function — infinite recursion. | ||
| type_obj = self.type_struct_name(derived_cl) | ||
| self.emit_line(f"{prefix}{type_obj}->tp_base->{tp_func}({args});") | ||
| base_var = f"_base_{tp_func}" | ||
| self.emit_line(f"PyTypeObject *{base_var} = {type_obj}->tp_base;") | ||
| self.emit_line(f"while ({base_var}->tp_flags & Py_TPFLAGS_HEAPTYPE) {{") | ||
| self.emit_line(f" {base_var} = {base_var}->tp_base;") | ||
| self.emit_line("}") | ||
| self.emit_line(f"{prefix}{base_var}->{tp_func}({args});") |
There was a problem hiding this comment.
it looks like with this change the __del__ method of an interpreted base class is skipped, for example in this test case:
[case testInheritedHeapBaseFinalizerRuns]
from mypy_extensions import mypyc_attr
events: list[str] = []
@mypyc_attr(native_class=False)
class HeapError(Exception):
def __del__(self) -> None:
events.append("base del")
class NativeError(HeapError):
def __init__(self, payload: object) -> None:
self.own = payload
[file driver.py]
import gc
from native import NativeError, events
for _ in range(1):
NativeError(object())
gc.collect()
assert events == ["base del"], events
i think we're missing a call to PyObject_CallFinalizerFromDealloc in generate_dealloc_for_class in the case for classes with builtin_base.
Sorry, something went wrong.
There was a problem hiding this comment.
Good catch, included this test as well, thank you!
Sorry, something went wrong.
When a mypyc-compiled Exception subclass inherits from an interpreted base that defines __del__, our custom tp_dealloc bypasses subtype_dealloc, so the inherited tp_finalize was never invoked. Runtime-gate on Py_TYPE(self)->tp_finalize and call PyObject_CallFinalizerFromDealloc for builtin_base classes.
There was a problem hiding this comment.
looks good, thanks!
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Fixes #21716
When mypyc compiles class Foo(Exception) with instance attributes, it staples an extra __dict__ slot onto the end of the base exception struct to hold them but it doesn't generate its own dealloc; the type inherits BaseExceptions, which only frees the base's own fields and has no idea the extra slot exists, meaning that the dict leaks on every destruction.
The fix: Flip the gate in emitclass.py so we emit our own tp_dealloc / tp_clear / tp_traverse for this case too. The existing builtin-base dealloc path then does the right thing: our clear frees the extra slot, then the base's dealloc frees the rest.