FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-29347: Fix possibly dereferencing undefined pointers when creating weakref objects by zhangyangyu · Pull Request #128 · python/cpython · GitHub

/ cpython Public

bpo-29347: Fix possibly dereferencing undefined pointers when creating weakref objects - #128

Merged
zhangyangyu merged 3 commits into
python:masterfrom
zhangyangyu:weakref-fix
Feb 20, 2017
Merged

bpo-29347: Fix possibly dereferencing undefined pointers when creating weakref objects#128
zhangyangyu merged 3 commits into
python:masterfrom
zhangyangyu:weakref-fix

Conversation

Copy link
Copy Markdown
Member

No description provided.

zhangyangyu added type-bug An unexpected behavior, bug, or error needs backport to 2.7 labels Feb 16, 2017
zhangyangyu merged commit d0e8212 into python:master Feb 20, 2017
zhangyangyu deleted the weakref-fix branch February 20, 2017 04:25
zhangyangyu added a commit to zhangyangyu/cpython that referenced this pull request Feb 20, 2017
zhangyangyu added a commit to zhangyangyu/cpython that referenced this pull request Feb 20, 2017

Copy link
Copy Markdown
Member Author

Thanks @methane and @berkerpeksag . I opened #186 , #187 , #188 to backport the patch.

zhangyangyu added a commit that referenced this pull request Feb 20, 2017
zhangyangyu added a commit that referenced this pull request Feb 20, 2017
zhangyangyu added a commit that referenced this pull request Feb 20, 2017
akruis referenced this pull request in stackless-dev/stackless Sep 5, 2017
akruis pushed a commit to akruis/cpython that referenced this pull request Sep 9, 2017
akruis pushed a commit to akruis/cpython that referenced this pull request Sep 9, 2017
Correct the name of the function __reduce_ex__(). And add a test case.

https://bitbucket.org/stackless-dev/stackless/issues/128
(grafted from 7db74e8ee4e77ad617dc5467da0efd3f3b7cf925)
akruis pushed a commit to akruis/cpython that referenced this pull request Oct 10, 2017
jaraco pushed a commit that referenced this pull request Dec 2, 2022
SonicField added a commit to SonicField/cpython that referenced this pull request Apr 24, 2026
Repro for re._parser JIT correctness regression discovered via
pythia python#128 python#2 perf seam (generalist 22:05Z; supervisor-opened
W-RE-PARSER-DEOPT-DETACH workstream 22:06:56Z).

Workload: cinderjit.auto() + repeated re.compile() of named-group
patterns past the 1000-call auto-compile threshold so
re._parser:Tokenizer.__next + SubPattern.__getitem__ + .getwidth
all hit deopt-backoff + JIT-detach. Subsequent re.compile()
either:

  (a) AttributeError: 'SubPattern' object has no attribute 'state'
      at Lib/re/_compiler.py:761 (p.state.groupdict)
  (b) SIGSEGV (exit 139, core dumped)

Without JIT_ENABLE=1: 2000+ iterations PASS clean.

Two failure modes (AttributeError + SIGSEGV) on the same workload
suggest UAF / corrupted attribute storage post-deopt-detach rather
than a missing __init__ run (theologian 22:06:38Z structural read).

JIT trace messages preceding crash include sequential Deopt
backoff for Tokenizer.__next, Tokenizer.match, SubPattern.__getitem__,
SubPattern.getwidth (all reaching 1000 guard failures and
detaching), with Skipping compilation messages for SubPattern.__init__
+ State.__init__.

Test is currently @unittest.skip per W-A4-mid sentinel precedent
(test exists, currently skipped, will be un-skipped when the
underlying bug is fixed and the workload runs clean — at that
point this becomes a regression gate).

cinderx_dev oracle deferred until devgpu004 SSH-2FA infra restored;
per Alex policy "all bugs presumed Phoenix-introduced" the working
hypothesis is Phoenix regression, not pre-existing CinderX class.

Auth: pythia python#128 python#2 surfacing; generalist 22:05:55Z bug
escalation; theologian 22:06:38Z + supervisor 22:06:56Z CONCUR (α)
+ sentinel commit + W-RE-PARSER-DEOPT-DETACH workstream open.
SonicField added a commit to SonicField/cpython that referenced this pull request Apr 24, 2026
…ntinel

Per docs/w-i3-runtime-assert-spec.md (theologian 21:57:02Z, supervisor
22:10:02Z, librarian 22:05:19Z framework guidance). Closes pythia
python#128 / python#131 python#1 named risk: I3 invariant (BasicBlock::id is allocation-
monotonic AND never mutated post-allocation) is grep-only, not
runtime-asserted.

Phase B B-gamma's PhxBcBlockArray (Tier 8 SECOND-PILOT, push 40
4145fe3) indexes by BasicBlock::id. A future HIR pass that
renumbers ids (SSA-destruction-style cache-locality renumbering,
peephole block-splitter, speculative-inlining clone) would silently
corrupt the dense-array lookup; I2 read-site check catches out-of-
range, NOT stale-mapping-correct-range.

(IV) CI grep gate — scripts/check_i3_invariant.sh + Step 1h wire:
- Detects "->id =", ".id =", set_id(/setId( patterns in Python/jit/hir/.
- Allow-list excludes legitimate non-BasicBlock matches:
  * HirLoadAttrSpecial::id (void* attribute identifier, different
    class)
  * next_register_id_, cache_id_ (different fields, trailing
    underscore)
  * test_*.c, hir_instr_c_verify.cpp (read-path testing, per W44
    ALLOW_LIST precedent)
- Falsification-tested: synthetic patch adding "block->id = 42;"
  to Python/jit/hir/synth_i3_violation.cpp triggers GATE FAIL EXIT=1
  under --strict; removing the patch returns to GATE PASS EXIT=0.
- Wired as Step 1h in scripts/gate_phoenix.sh per librarian 22:05:19Z
  framework guidance, mirroring Steps 1e/1g delegation shape (capture
  OUTPUT, tee to RESULTS_FILE, GATE_PASS=0 + FAILURES on nonzero).

(III) JIT_DCHECK pydebug sentinel — PhxBcBlockEntry.sentinel_id:
- Adds int sentinel_id field under #ifdef Py_DEBUG to PhxBcBlockEntry
  (zero release-build cost; pydebug entry size 8 -> 12 bytes).
- Insert sets sentinel_id = block_id at insert time.
- Lookup verifies sentinel_id == block_id via JIT_DCHECK_C; mismatch
  fires JIT_CHECK_C abort with diagnostic indicating either a
  post-allocation id mutation OR inserter/reader index disagreement.
- catches the (IV) miss-class: id mutation via member function /
  indirect pointer write that grep doesn't pattern-match.

Combined cost: zero release runtime; ~45min implementation. (I)
sentinel-always and (II) generation-counter REJECTED per spec §3 as
reverse perf trade without strong evidence the always-on detection
is required vs (IV)+(III) catching at commit + pydebug.

Build implications: Py_DEBUG-conditional layout change to
PhxBcBlockEntry. Per JIT Header Change Protocol (CLAUDE.md), requires
full distclean rebuild; testkeeper to verify both release (sentinel
field absent, struct == 8 bytes) and pydebug (sentinel field present,
struct == 12 bytes, JIT_DCHECK_C does not fire on normal workload).
ARM64 verification via standard ARM64 retroactive-debt path once
devgpu004 SSH-2FA infra restored.

Auth: theologian spec 21:57:02Z + supervisor CONCUR 21:57:20Z +
22:10:02Z; librarian framework guidance 22:05:19Z (Step 1h shape
mirroring 1e/1g, scripts/check_i3_invariant.sh --strict).
SonicField added a commit to SonicField/cpython that referenced this pull request Apr 25, 2026
Per theologian W27c pair-batch design 00:08:52Z + 16:57:48Z
APPROVE; supervisor (α) local-accumulate posture. Conversion #2b
in the 5-method PARTIAL backlog (now 3 → 2 PARTIAL Cat-B
remaining); pair-batch sibling of W27c #2a (7135d94).

Cat B residue elimination — pair-batch closure: this commit
mirrors W27c #2a emitInlineExceptionMatch's shape and reuses the
same C helper build_inline_except_opcode_array_c (introduced in
#2a). The pre-resolve loop that previously ran on the C++ side
(BytecodeInstruction iteration + PyTuple_GET_ITEM(co_consts) for
LOAD_CONST/RETURN_CONST + getBlockAtOff for JUMP_BACKWARD* + the
terminator-detection skip-list) is deleted C++-side and replaced
with a single helper invocation in the C body.

Signature change: hir_builder_emit_call_exception_handler_c drops
(opcodes, opcode_count) params; builds opcode array internally
from PyCodeObject (via tc->frame.code) + builder block_map +
except_body_offset, identical to the #2a entry path. C++ stub
shrinks from ~70 lines to ~25.

D1 pre-amble (call_instr->setSuppressExceptionDeopt(true) +
phx_ptr_arr_pop on tc.frame.stack) stays C++-side because
DeoptBase*/Register* are C++-only types — that boundary is
intentional and unchanged.

D3 ok-block sequence (RefineType in-place SSA-rename + push
result back to outer tc stack to undo emitAnyCall's pre-amble
pop) preserved verbatim.

Bridge-struct elimination (bonus cleanup): with both #2a and #2b
now C-side for opcode-array construction, the C++/C bridge struct
OpcodeArrayEntry_CXX (+ its static_assert) in builder.cpp has
zero remaining references. Deleted entirely. The C-side
OpcodeArrayEntry struct in builder_emit_c.c stays (still used by
emit_except_match_body_c). This realises the design intent stated
in W27c #2a's commit message ("Eliminates OpcodeArrayEntry C++/C
bridge struct entirely").

Honest ratio framing per librarian D-1776879377 / D-1776880214:
post-conversion 98/100 PURE-CONVERTED + 2 PARTIAL Cat-B remaining
(was 97/100 + 3 PARTIAL after W27c #2a). emitCallExceptionHandler
is now a delegation stub, not pure-C++-eliminated; same metric
reading as the established honest-scope record. Remaining 2
PARTIAL are emitAnyCall (await-tail residue, design-residual per
pythia python#128 D-1777029012) and emitLoadMethodStatic (classloader.h
_PyType_VTable struct dependency, design-residual per A1 rule).

VERIFICATION: deferred to testkeeper full gate cycle (post-reboot
fresh verifications per supervisor 16:54:08Z stale-pre-reboot
discipline). All pre-reboot results (testkeeper 00:48:33Z +
01:45:23Z + 05:39:16Z) RETIRED. Push 54 batch (W27c python#1
7bfb7d1 + #2a 7135d94 + this commit) gated on:
- x86_64 fresh build + JIT smoke + nbody + W-RE-PARSER
  substrate-stability + Phoenix CPython suite + ABBA 4-bench
- ARM64 STRICT verify on 3-commit bundle (devgpu004 connectivity
  restored 16:53Z post-server-reboot)
- gatekeeper APPROVE
- supervisor PUSH AUTHORIZED

Auth chain: theologian design 00:08:52Z + APPROVE 16:57:48Z;
supervisor (α) authorization 00:07:10Z + 16:55:47Z plan; testkeeper
standby for fresh full gate (post-reboot discipline).
SonicField added a commit to SonicField/cpython that referenced this pull request Apr 25, 2026
Per theologian W27c pair-batch design 00:08:52Z + 16:57:48Z
APPROVE; supervisor (α) local-accumulate posture. Conversion #2b
in the 5-method PARTIAL backlog (now 3 → 2 PARTIAL Cat-B
remaining); pair-batch sibling of W27c #2a (7135d94).

Cat B residue elimination — pair-batch closure: this commit
mirrors W27c #2a emitInlineExceptionMatch's shape and reuses the
same C helper build_inline_except_opcode_array_c (introduced in
#2a). The pre-resolve loop that previously ran on the C++ side
(BytecodeInstruction iteration + PyTuple_GET_ITEM(co_consts) for
LOAD_CONST/RETURN_CONST + getBlockAtOff for JUMP_BACKWARD* + the
terminator-detection skip-list) is deleted C++-side and replaced
with a single helper invocation in the C body.

Signature change: hir_builder_emit_call_exception_handler_c drops
(opcodes, opcode_count) params; builds opcode array internally
from PyCodeObject (via tc->frame.code) + builder block_map +
except_body_offset, identical to the #2a entry path. C++ stub
shrinks from ~70 lines to ~25.

D1 pre-amble (call_instr->setSuppressExceptionDeopt(true) +
phx_ptr_arr_pop on tc.frame.stack) stays C++-side because
DeoptBase*/Register* are C++-only types — that boundary is
intentional and unchanged.

D3 ok-block sequence (RefineType in-place SSA-rename + push
result back to outer tc stack to undo emitAnyCall's pre-amble
pop) preserved verbatim.

Bridge-struct elimination (bonus cleanup): with both #2a and #2b
now C-side for opcode-array construction, the C++/C bridge struct
OpcodeArrayEntry_CXX (+ its static_assert) in builder.cpp has
zero remaining references. Deleted entirely. The C-side
OpcodeArrayEntry struct in builder_emit_c.c stays (still used by
emit_except_match_body_c). This realises the design intent stated
in W27c #2a's commit message ("Eliminates OpcodeArrayEntry C++/C
bridge struct entirely").

Honest ratio framing per librarian D-1776879377 / D-1776880214:
post-conversion 98/100 PURE-CONVERTED + 2 PARTIAL Cat-B remaining
(was 97/100 + 3 PARTIAL after W27c #2a). emitCallExceptionHandler
is now a delegation stub, not pure-C++-eliminated; same metric
reading as the established honest-scope record. Remaining 2
PARTIAL are emitAnyCall (await-tail residue, design-residual per
pythia python#128 D-1777029012) and emitLoadMethodStatic (classloader.h
_PyType_VTable struct dependency, design-residual per A1 rule).

VERIFICATION: deferred to testkeeper full gate cycle (post-reboot
fresh verifications per supervisor 16:54:08Z stale-pre-reboot
discipline). All pre-reboot results (testkeeper 00:48:33Z +
01:45:23Z + 05:39:16Z) RETIRED. Push 54 batch (W27c python#1
7bfb7d1 + #2a 7135d94 + this commit) gated on:
- x86_64 fresh build + JIT smoke + nbody + W-RE-PARSER
  substrate-stability + Phoenix CPython suite + ABBA 4-bench
- ARM64 STRICT verify on 3-commit bundle (devgpu004 connectivity
  restored 16:53Z post-server-reboot)
- gatekeeper APPROVE
- supervisor PUSH AUTHORIZED

Auth chain: theologian design 00:08:52Z + APPROVE 16:57:48Z;
supervisor (α) authorization 00:07:10Z + 16:55:47Z plan; testkeeper
standby for fresh full gate (post-reboot discipline).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants


Back | FazBrowse Home | New Git URL