FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-124697: avoid duplicate names in the same frame by iritkatriel · Pull Request #158924 · python/cpython · GitHub

Repository navigation

Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension .c  (5) .h  (6) .md  (1) .py  (2) .rst  (1) All 5 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
10 changes: 10 additions & 0 deletions Doc/library/dis.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -2003,6 +2003,16 @@ but are replaced by real opcodes or removed before bytecode is generated.
.. versionchanged:: 3.13
This opcode is now a pseudo-instruction.

.. opcode:: STORE_CLOSURE (i)

Stores the TOS into the cell slot ``i`` of the "fast locals" storage.
Used to restore a cell saved by ``LOAD_CLOSURE`` when isolating an
inlined comprehension that reuses an enclosing free variable.

Note that ``STORE_CLOSURE`` is replaced with ``STORE_FAST`` in the assembler.

.. versionadded:: next


.. _opcode_collections:

Expand Down
6 changes: 6 additions & 0 deletions Include/internal/pycore_interpframe.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,12 @@ _PyFrame_Traverse(_PyInterpreterFrame *frame, visitproc visit, void *arg);
bool
_PyFrame_HasHiddenLocals(_PyInterpreterFrame *frame);

/* True when localsplus[oparg] is a free cell that currently differs from
* the function's func_closure cell — i.e. an inlined comprehension has
* temporarily replaced it. */
PyAPI_FUNC(bool)
_PyFrame_IsInlinedCompTempFree(_PyInterpreterFrame *frame, int oparg);

PyObject *
_PyFrame_GetLocals(_PyInterpreterFrame *frame);

Expand Down
24 changes: 16 additions & 8 deletions Include/internal/pycore_opcode_metadata.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion Include/opcode_ids.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

49 changes: 38 additions & 11 deletions InternalDocs/inlined_comprehensions.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -86,15 +86,35 @@ The walk stops at a class: nested scopes do not see class locals.
Class-closure names that would otherwise be free through a class become
`GLOBAL_IMPLICIT`.

If the inlined name is `LOCAL` or `CELL` but the nearest non-inlined
enclosing table has it as `FREE` (or `DEF_FREE_CLASS`), resolve it as
`FREE` so the comprehension reuses that localsplus slot.
`compiler_cellvars()` also skips adding those child cells, which would
otherwise create a second same-named entry. Class-closure names
(`__class__` and friends) are never reused: zero-arg `super()` and
class-cell bookkeeping need the real free cell.

### Isolating iteration variables

`codegen_push_inlined_comprehension_locals()` in
[`Python/codegen.c`](../Python/codegen.c) isolates names bound in the
comprehension:
[`Python/codegen.c`](../Python/codegen.c) isolates each name bound in
the comprehension on one of two paths:

Reuse an enclosing free (`_PyCompile_GetRefType()` is `FREE`):

* `LOAD_CLOSURE` saves the enclosing cell without clearing the slot
(so free-slot introspection never sees a NULL).
* `MAKE_CELL` on a `CO_FAST_FREE` slot always installs a fresh empty
cell, replacing the saved one. The comprehension then uses `DEREF`.
* Restore uses `STORE_CLOSURE` (a cell/free-index pseudo that becomes
`STORE_FAST` after `fix_cell_offsets`).

* `LOAD_FAST_AND_CLEAR` saves the enclosing value (possibly `NULL`) and
clears the slot.
* `MAKE_CELL` runs if the name is a cell for this comprehension.
Own fast-local slot (everything else):

* `LOAD_FAST_AND_CLEAR` saves the enclosing value (possibly `NULL`)
and clears the slot.
* `MAKE_CELL` runs only if the name is a cell for this comprehension.
* Restore uses `STORE_FAST_MAYBE_NULL`.
* In module and class units the name is added to `u_fasthidden` so
assemble can set `CO_FAST_HIDDEN`.

Expand All @@ -105,10 +125,19 @@ or `finally` sees the original values.
Runtime
-------

An inlined comprehension cell can share a localsplus name with an
enclosing free variable (for example `[lambda: x for x in x]` inside a
nested function). `FrameLocalsProxy` keys, values, items, and `len`
keep the first slot of each name so they agree with `getitem`.
An inlined comprehension local that collides with an enclosing free
(for example `[x for x in x]` or `[lambda: x for x in x]` inside a
nested function) reuses the free slot. Isolation saves that cell and
installs a temporary one so `STORE_DEREF` does not change the value
seen by existing closures; lambdas that capture the iteration variable
share the temporary cell. After the comprehension, the original cell
is restored.

While the temporary cell is installed, `_PyFrame_IsInlinedCompTempFree()`
is true (frame cell differs from `func_closure`). That drives
class/module `locals()` to use `FrameLocalsProxy` even without a
`CO_FAST_HIDDEN` slot, and makes an empty temporary cell raise
`UnboundLocalError` rather than `NameError`.

Source
------
Expand All @@ -128,5 +157,3 @@ Source
`InlinedComprehensionBlock`
* [`Include/internal/pycore_compile.h`](../Include/internal/pycore_compile.h):
`_PyCompile_InlinedComprehensionState`
* [`Objects/frameobject.c`](../Objects/frameobject.c):
`FrameLocalsProxy` duplicate-name handling
3 changes: 2 additions & 1 deletion Lib/_opcode_metadata.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading

Back | FazBrowse Home | New Git URL