C Extension Analysis Report

  Extension: msgpack._cmsgpack

  Scope: entire project

  Agents Run: type-slot-checker, gil-discipline-checker, resource-lifecycle-checker,
  module-state-checker, parity-checker, version-compat-scanner, c-complexity-analyzer,
  stable-abi-checker, git-history-analyzer

  (refcount-auditor, error-path-analyzer, null-safety-scanner, pyerr-clear-auditor
  skipped — Cython-mode, validated zero FIX-class loss)

  External Tools: cppcheck (7 .h files scanned), clang-tidy (skipped — no
  compile_commands.json; requires `make cython && python setup.py build_ext -i` first)

  ---
  Executive Summary

  The extension is broadly well-implemented for a Cython project: per-instance buffer
  allocation, comprehensive @cython.critical_section coverage on public methods, and a
  solid exception-handling architecture. Three areas need immediate attention. First,
  the recent free-threading commit (#641) missed Packer.__getbuffer__ and
  __releasebuffer__ — the two buffer-protocol methods that directly protect the C
  buffer from concurrent reallocation — creating a use-after-free window under
  Py_GIL_DISABLED. Second, Unpacker.__init__ has a pre-existing re-initialization leak
  pattern (both the C buffer and the in-progress parse stack) that has never been fixed
  upstream. Third, the pure-Python fallback diverges from the Cython implementation in
  three user-visible ways: read_bytes() return type, object_pairs_hook contract, and
  skip() exception translation. The git history also reveals the fork is 7+ bug-fix
  commits behind the upstream canonical repo.

  ---
  Extension Profile

  - Module: msgpack._cmsgpack (3 .pyx files, 940 lines; 7 hand-written .h files, 1,830
  lines)
  - Init style: Multi-phase (structural), but with process-global module state
  (CYTHON_USE_MODULE_STATE=0, m_size=0)
  - Python targets: ≥3.10
  - Limited API: not claimed
  - Code generation: Cython 3.2.1
  - freethreading_compatible = True declared

  ---
  External Tool Baseline

  cppcheck ran against all 7 hand-written .h files (pack.h, pack_template.h,
  sysdep.h, unpack.h, unpack_container_header.h, unpack_define.h,
  unpack_template.h). One finding:

  ▎ FALSE POSITIVE — pack_template.h:256 (cppcheck: syntaxError)
  ▎ Root cause: cppcheck's preprocessor evaluator cannot resolve the
  ▎ three-branch portability idiom used in msgpack_pack_short (and
  ▎ msgpack_pack_int, msgpack_pack_long, etc.):
  ▎
  ▎   #if defined(SIZEOF_SHORT)
  ▎     #if SIZEOF_SHORT == 2 ... #endif
  ▎   #elif defined(SHRT_MAX)
  ▎     #if SHRT_MAX == 0x7fff ... #endif  ← nested #if inside #elif
  ▎   #else
  ▎     if(sizeof(short) == 2) { ... }    ← plain if, not #if
  ▎   #endif
  ▎
  ▎ The code is valid C. cppcheck bails at line 256 and does not analyze
  ▎ the remainder of pack_template.h (~350 lines including _PyFloat_* guards
  ▎ and the unsigned-int packing logic). See also finding 23 (complexity).
  ▎
  ▎ Workaround: build the extension with `make cython && python setup.py
  ▎ build_ext -i`, then pass `compile_commands.json` to cppcheck via
  ▎ `--project=`. clang-tidy with compile_commands.json would also
  ▎ resolve the macro definitions and produce richer analysis.

  ---
  Key Metrics

  ┌──────────────┬────────┬─────┬──────────┬───────────────────────────────────────┐
  │  Dimension   │ Status │ FIX │ CONSIDER │              Top Finding              │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Refcount     │ G      │ 0   │ 0        │ Cython-mode skip                      │
  │ Safety       │        │     │          │                                       │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Error        │ G      │ 0   │ 0        │ Cython-mode skip                      │
  │ Handling     │        │     │          │                                       │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ NULL Safety  │ G      │ 0   │ 0        │ Cython-mode skip                      │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ GIL          │ Y      │ 1   │ 0        │ __getbuffer__/__releasebuffer__       │
  │ Discipline   │        │     │          │ missing @cython.critical_section      │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Resource     │ G      │ 0   │ 1        │ Partial parse-stack retained on       │
  │ Lifecycle    │        │     │          │ _unpack error paths                   │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Module State │ Y      │ 0   │ 3        │ 5 process-global PyObject* statics;   │
  │              │        │     │          │ PyDateTimeAPI cross-interpreter       │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Type Slots   │ Y      │ 2   │ 0        │ Unpacker.__init__ re-init leaks       │
  │              │        │     │          │ self.buf and parse stack              │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ PyErr_Clear  │ G      │ 0   │ 0        │ Cython-mode skip                      │
  │ Safety       │        │     │          │                                       │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ ABI          │ G      │ 0   │ 6        │ Not claimed; datetime C API is hard   │
  │ Compliance   │        │     │          │ blocker                               │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Version      │ G      │ 0   │ 5        │ Dead sysdep.h guards; _PyFloat_*      │
  │ Compat       │        │     │          │ correctly guarded to 3.10-only        │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ C/Python     │ Y      │ 3   │ 5        │ read_bytes type, object_pairs_hook    │
  │ Parity       │        │     │          │ contract, skip() exception            │
  ├──────────────┼────────┼─────┼──────────┼───────────────────────────────────────┤
  │ Complexity   │ G      │ 0   │ 3        │ unpack_execute CC ~60+, macro/goto    │
  │              │        │     │          │ contract undocumented                 │
  └──────────────┴────────┴─────┴──────────┴───────────────────────────────────────┘

  G = 0 FIX | Y = 1–3 FIX | R = 4+ FIX

  ---
  Findings by Priority

  Must Fix (FIX) — 6

  #: 1
  Finding: Packer.__getbuffer__ and __releasebuffer__ missing @cython.critical_section
  —
    concurrent pack() can call PyMem_Realloc on self.pk.buf while another thread's
    memoryview(packer) is calling PyBuffer_FillInfo on the same pointer;
  use-after-free.
     Also self.exports read-modify-write is non-atomic.
  File:Line: _packer.pyx:363-368
  Agents: gil-discipline, git-history
  ────────────────────────────────────────
  #: 2
  Finding: Unpacker.__init__ overwrites self.buf via PyMem_Malloc without first freeing

    the existing buffer — any re-call to __init__ (or subclass super().__init__(...))
    leaks the prior allocation
  File:Line: _unpacker.pyx:373
  Agents: type-slot, git-history
  ────────────────────────────────────────
  #: 3
  Finding: Unpacker.__init__ calls unpack_init (not unpack_clear) before
  re-initializing
    the parse context — live PyObject* references in ctx->stack[1..top] are leaked if
    called on a mid-stream Unpacker
  File:Line: _unpacker.pyx:385
  Agents: type-slot, git-history
  ────────────────────────────────────────
  #: 4
  Finding: fallback.Unpacker.read_bytes() returns bytearray (mutable, unhashable);
    Cython returns bytes — dict-key use, isinstance checks, and hashability all diverge
  File:Line: fallback.py:346-348
  Agents: parity, git-history
  ────────────────────────────────────────
  #: 5
  Finding: fallback.Unpacker passes a generator to object_pairs_hook; Cython passes a
    fully-materialised list — len(pairs), re-iteration, and subscript access all break;

    additionally, strict_map_key validation is lazy in the fallback and can be bypassed

    by a hook that doesn't fully consume the generator
  File:Line: fallback.py:520-529
  Agents: parity, git-history
  ────────────────────────────────────────
  #: 6
  Finding: fallback.Unpacker.skip() lets RecursionError propagate raw; every other
    unpack method converts it to StackError — callers catching StackError/ValueError to

    guard against adversarial nested data miss the exception on this path
  File:Line: fallback.py:584-587
  Agents: parity, git-history

  ---
  Should Consider (CONSIDER) — 20

  #: 7
  Finding: Unpacker._unpack error paths (FormatError, StackError, PyErr_Occurred) do
  not
    call unpack_clear before raising — partially-built container objects in
    ctx->stack[1..top] are retained until __dealloc__. Bounded at 1024 slots; contrast
    with unpackb which already calls unpack_clear on failure
  File:Line: _unpacker.pyx:484-491
  ────────────────────────────────────────
  #: 8
  Finding: utc, epoch, giga, ExtType, Timestamp compile to file-scope static PyObject*
    process globals (CYTHON_USE_MODULE_STATE=0). Re-importing in a second
  subinterpreter
     clobbers the shared pointer, leaving the first interpreter's live Packer/Unpacker
    instances with stale or cross-interpreter object references
  File:Line: _cmsgpack.pyx:8-9, _unpacker.pyx:20
  ────────────────────────────────────────
  #: 9
  Finding: PyDateTimeAPI is a per-translation-unit static pointer (from datetime.h).
  The
    module-exec function calls import_datetime() per interpreter import, clobbering the

    global with a capsule from the new interpreter. Cross-interpreter datetime
  timestamp
     decode (timestamp=3) will call function pointers from the wrong interpreter
  File:Line: _cmsgpack.pyx:5, unpack.h:339-344
  ────────────────────────────────────────
  #: 10
  Finding: Py_mod_multiple_interpreters: NOT_SUPPORTED slot is guarded by #if
    CYTHON_USE_MODULE_STATE (always 0 in shipped builds) — the multi-interpreter
  posture
     is absent from the compiled binary, leaving CPython to apply an implicit default
    rather than an explicit declaration
  File:Line: generated C
  ────────────────────────────────────────
  #: 11
  Finding: Fork is 7+ upstream bug-fix commits behind: missing pack_timestamp
    return-code checking (378edc6), missing pack_ext_type autoreset (284782d), missing
    unpack_callback_int64 null check (95c8be5), missing unpack_container_header return
    check (156bb05)
  File:Line: various
  ────────────────────────────────────────
  #: 12
  Finding: unpackb accepts max_buffer_size and read_size silently in the fallback (via
    **kwargs to Unpacker.__init__) but raises TypeError in Cython — a max_buffer_size
    limit passed to unpackb is silently ignored by the Cython path, breaking any caller

    relying on it as a security cap
  File:Line: _unpacker.pyx:141-149, fallback.py:72
  ────────────────────────────────────────
  #: 13
  Finding: pack_ext_type in Cython passes typecode directly to a C char — values
  outside
    0–127 silently overflow or encode the reserved timestamp type (-1); the fallback
    validates and raises ValueError for out-of-range codes
  File:Line: _packer.pyx:288-297
  ────────────────────────────────────────
  #: 14
  Finding: Multi-byte buffer input raises BufferError in Cython, ValueError in fallback

    — callers catching ValueError to handle all invalid input miss the exception on the

    Cython path
  File:Line: _unpacker.pyx:127, fallback.py:68
  ────────────────────────────────────────
  #: 15
  Finding: read_size default is 1 MiB in Cython vs 16 KiB in fallback and docs — Cython

    issues 64× larger reads than documented from file_like sources
  File:Line: _unpacker.pyx:369
  ────────────────────────────────────────
  #: 16
  Finding: datetime subclass packing: Cython uses PyDateTime_CheckExact (excludes
    subclasses like pendulum.DateTime), fallback uses isinstance (includes them) —
    undocumented behavioral split
  File:Line: _packer.pyx:243
  ────────────────────────────────────────
  #: 17
  Finding: Dead MSVC pre-2010 integer typedef block (_MSC_VER < 1600) — unreachable
    since VS 2010; Python ≥3.10 requires VS 2019+
  File:Line: sysdep.h:23-31
  ────────────────────────────────────────
  #: 18
  Finding: Dead GCC <4.1 atomic macro branch + unused
    _msgpack_atomic_counter_t/_msgpack_sync_* infrastructure — defined but never called

    anywhere in the codebase; likely a leftover from an earlier refcount-based design
  File:Line: sysdep.h:39-49
  ────────────────────────────────────────
  #: 19
  Finding: pyproject.toml declares cython>=3.2.5 with no upper bound; CI pins
    Cython==3.2.1 via requirements.txt — developer environments can pick up Cython 4
    before any migration review
  File:Line: pyproject.toml:52, requirements.txt:1
  ────────────────────────────────────────
  #: 20
  Finding: PyUnicode_InternInPlace on every map key from untrusted sources — in Python
    3.12+ mortal interned strings persist in the intern table; adversarial input can
    hold arbitrarily many unique strings in the table, a memory amplification vector
  File:Line: unpack.h:191-192
  ────────────────────────────────────────
  #: 21
  Finding: Py_SIZE(o) used on bytes/bytearray/list — soft-deprecated direct struct
    access; stable-ABI-safe alternatives (PyBytes_Size, PyList_Size, etc.) are drop-in
    replacements
  File:Line: _packer.pyx:186,199,226
  ────────────────────────────────────────
  #: 22
  Finding: Py_TYPE(o)->tp_name direct PyTypeObject field dereference in error-path
    format strings — not available under Limited API; use %T (3.12+) or PyType_GetName
    (3.11+)
  File:Line: _packer.pyx:188,254,256, unpack.h:188
  ────────────────────────────────────────
  #: 23
  Finding: unpack_execute (~308 lines, CC ~60+, 10 goto targets) — the macro/goto
    contract is undocumented; adding a new msgpack type without knowing that every case

    arm must terminate with a push_*/again_* macro will silently fall through to
    _failed. Also: MSVC and GCC code paths for SWITCH_RANGE diverge without CI coverage

    of the MSVC path
  File:Line: unpack_template.h:86-394
  ────────────────────────────────────────
  #: 24
  Finding: _pack_inner (~105 lines, CC ~25) — unicode encoding and datetime arithmetic
    are inlined; extracting _pack_unicode and _pack_datetime helpers would reduce
    cognitive load without performance cost
  File:Line: _packer.pyx:152-256
  ────────────────────────────────────────
  #: 25
  Finding: unpack_callback_ext (~90 lines, CC ~20) — four timestamp-mode arms each
    manage 2–5 manual refcount variables with early-return error paths; variable
    aliasing in mode-1 (float path: a is reassigned after Py_DECREF calls); extracting
    per-mode helpers would remove this as the highest-risk refcount surface in the
    codebase
  File:Line: unpack.h:292-381
  ────────────────────────────────────────
  #: 26
  Finding: PyList_SET_ITEM/PyTuple_SET_ITEM are non-Limited-API macros (direct ob_item
    access); PyList_SetItem/PyTuple_SetItem are stable-ABI drop-ins (add return-value
    check)
  File:Line: unpack.h:148,150,200

  ---
  Tensions

  - Free-threading vs. subinterpreters: The extension declared freethreading_compatible
  = True in the last commit but has process-global PyObject* state (findings 8–10)
  that makes it subinterpreter-unsafe. These goals are not contradictory —
  free-threading safety is per-instance lock discipline, subinterpreter safety is
  per-interpreter state isolation — but shipping freethreading_compatible without the
  matching NOT_SUPPORTED slot risks user expectation mismatch.
  - strict_map_key security in object_pairs_hook: The Cython path enforces key
  validation eagerly in C before the hook is called (safe). The fallback enforces it
  lazily inside a generator (finding 5). Upstream committed a partial fix (cd81370) but
  it still passes a generator, not a list.

  ---
  Policy Decisions (POLICY) — 3

  #: P1
  Finding: _PyFloat_Pack4/8/_PyFloat_Unpack4/8 private API usage is correctly guarded
  to
    Python 3.10 only via #if PY_VERSION_HEX >= 0x030B00A7. When Python 3.10 reaches EOL

    (Oct 2026), remove the #else branches entirely. Alternatively, replace with manual
    IEEE 754 packing using the existing _msgpack_store32/64/_msgpack_load32/64
  byte-swap
     infrastructure — this eliminates the version fork and the CPython dependency from
    the float path.
  ────────────────────────────────────────
  #: P2
  Finding: Stable ABI (abi3) is not feasible at Python 3.10 without rework — the hard
    blockers are the datetime C API (PyDelta_FromDSU not in stable ABI, PyDateTimeAPI
    capsule not in Limited API) and _PyFloat_* on Python 3.10. Raising the minimum to
    3.11 fixes the float issue; the datetime path requires replacing capsule calls with

    PyObject_CallMethod/PyObject_IsInstance. Benefit: single abi3 wheel replacing
    per-minor wheels. Assess when preparing the 3.10-EOL release.
  ────────────────────────────────────────
  #: P3
  Finding: Add an unconditional {Py_mod_multiple_interpreters,
    Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED} slot to the module def (not guarded by
    CYTHON_USE_MODULE_STATE) to make the multi-interpreter posture explicit. This is a
    near-zero-cost protective declaration that blocks CPython from loading the module
    into a second subinterpreter until the process-global state (findings 8–9) is
    resolved.

  ---
  Strengths

  - Per-instance locking discipline is excellent: Every public Python method on Packer
  and Unpacker carries @cython.critical_section, correctly serialising per-object state
  for free-threading. The two missing methods (finding 1) are a narrow, fixable gap in
  an otherwise complete implementation.
  - C buffer management is correct on the happy path: msgpack_pack_write uses the
  classic local-pointer-then-commit realloc pattern (pack.h:47-51) that correctly
  preserves pk->buf on allocation failure. All 4 tracked heap allocations across the
  extension have matching frees.
  - Exception propagation is thorough: unpackb and Unpacker._unpack both catch and
  correctly convert RecursionError, OutOfData, FormatError, and re-raise existing
  exceptions — a complete chain validated by the recent upstream sweep.
  - No _Py* private API beyond the correctly-version-guarded _PyFloat_* fallback: The
  extension uses only stable, documented CPython APIs in production code paths.
  - unpack_callback_ext and the streaming FSM are functionally correct: Despite their
  complexity, no correctness bugs were found in unpack_execute or the timestamp decode
  path during this review.
  - freethreading_compatible declaration is substantively justified: Module-level
  globals are write-once at import and read-only thereafter; per-instance C buffers are
  strictly isolated. The declaration is correct subject to fixing finding 1.

  ---
  Code Removal Opportunities

  Based on data/deprecated_apis.json patterns applicable at Python ≥3.10:

  - Py_SIZE → PyBytes_Size/PyList_Size/PyTuple_Size: 3 call sites
  (_packer.pyx:186,199,226) — mechanical one-line substitution each.
  - Py_TYPE()->tp_name → %T format (Python ≥3.12): 4 sites (_packer.pyx:188,254,256,
  unpack.h:188) — simplifies error formatting, removes struct access.
  - Dead sysdep.h blocks (findings 17–18): Removing the MSVC pre-2010 typedefs and the
  unused atomic counter infrastructure eliminates ~25 lines of dead code.
  - _PyFloat_* fallback (when 3.10 EOL): Removing the #else branches in pack_template.h
  and unpack_template.h eliminates ~8 lines and the last private-API usage.

  ---
  Recommended Action Plan

  Immediate (FIX items)

  1. Fix Finding 1 — Add @cython.critical_section to Packer.__getbuffer__ and
  __releasebuffer__ (_packer.pyx:363-368). Two lines.
  2. Fix Findings 2 & 3 — At the top of Unpacker.__init__, add a guard: if self.buf !=
  NULL: unpack_clear(&self.ctx); PyMem_Free(self.buf); self.buf = NULL before the
  PyMem_Malloc call (_unpacker.pyx:373).
  3. Fix Finding 4 — Change fallback.Unpacker.read_bytes to wrap result in bytes(...)
  before returning (fallback.py:348).
  4. Fix Finding 5 — Materialise the key-value list before calling object_pairs_hook in
  the fallback (fallback.py:520-529), and enforce strict_map_key eagerly on each key.
  5. Fix Finding 6 — Wrap fallback.Unpacker.skip() body in try/except RecursionError:
  raise StackError (fallback.py:584-587).
  6. Finding 11 — Sync with upstream canonical repo to pick up the 7 post-#641 bug-fix
  commits; priority: 378edc6 (pack_timestamp return codes), 95c8be5
  (unpack_callback_int64 null check), 284782d (pack_ext_type autoreset).

  Short-term (CONSIDER items)

  7. Finding 7 — Add unpack_clear(&self.ctx) before each raise in Unpacker._unpack
  (mirroring unpackb).
  8. Finding 13 — Add typecode range validation (0–127) to Packer.pack_ext_type in the
  Cython path.
  9. Finding 15 — Fix read_size default in _unpacker.pyx:369 to min(max_buffer_size,
  16*1024).
  10. Finding 23 — Add a block comment to unpack_execute documenting the macro/goto
  contract ("every case arm must terminate with a push_*/again_* macro"). Add a CI
  build that tests the MSVC code path for SWITCH_RANGE parity.
  11. Findings 17–18 — Remove dead MSVC <1600 typedef block and unused atomic counter
  infrastructure from sysdep.h.
  12. Finding 19 — Add <4 upper bound to Cython dev dependency in pyproject.toml; align
  with requirements.txt.
  13. Finding P3 — Add unconditional Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED slot.

  Longer-term (POLICY)

  14. Findings 8–10 / P3 — Build with -DCYTHON_USE_MODULE_STATE=1 to activate Cython
  3.2+'s per-interpreter module state, migrating the 5 cdef object globals into
  __pyx_mstatetype. This is the path to genuine subinterpreter safety.
  15. Finding 25 / P1 — When Python 3.10 reaches EOL: collapse _PyFloat_* #else
  branches; extract unpack_callback_ext timestamp modes into per-mode helpers.
  16. Finding P2 — When preparing the 3.10-EOL release: assess abi3 migration; the
  datetime path rework is the main investment.

  ---

  ▎ Note before sharing externally: This report is an internal artifact. Before filing
  ▎ issues or opening PRs against the upstream msgpack/msgpack-python repo, review
  ▎ WORKING_WITH_MAINTAINERS.md (../../../WORKING_WITH_MAINTAINERS.md). The upstream
  ▎ maintainer is actively fixing bugs (7 commits since #641); reach out first to
  ▎ understand preferred contribution format. Security-relevant findings (finding 5's
  ▎ strict_map_key bypass via generator) should go through responsible disclosure if
  ▎ reported publicly.

  Free-Threading Analysis Report

  Extension: msgpack._cmsgpack (3 .pyx files, 7 .h files, ~2,770 lines)

  Agents Run: shared-state-auditor, ft-history-analyzer, lock-discipline-checker,
  atomic-candidate-finder, unsafe-api-detector, stop-the-world-advisor

  (tsan-report-analyzer skipped — no TSan report provided;
   stw-safety-checker skipped — no _PyEval_StopTheWorld usage)

  ---
  Migration Status

  - First ft commit: af45640 "cython: freethreading_compatible (#654)" — 2025-10-09
  - Latest ft commit: 9de2fd9 "Add no-GIL interpreter support (#641)" — 2026-06-02
  - Substantive migration commits: 2 (af45640 adds critical sections; 9de2fd9 adds FT CI)

  Decorator sweep (af45640):
  - Packer public methods decorated: __init__, pack, pack_ext_type,
    pack_array_header, pack_map_header, pack_map_pairs, reset, bytes (8/10)
  - Packer public methods missed: __getbuffer__, __releasebuffer__ (2/10)
  - Unpacker public methods decorated: __init__, feed, read_bytes, unpack, skip,
    read_array_header, read_map_header, tell, __next__ (9/9)

  CI infrastructure (9de2fd9):
  - Adds pytest-run-parallel with --parallel-threads=auto --iterations=20 for *t
    Python versions
  - Adds test/uneeded_test_multithreading.py (manual stress test for Packer.pack)
    NOTE: this file is NEVER collected by pytest — filename does not match
    test_*.py or *_test.py due to the "uneeded" typo. The stress test silently
    does not run in CI.

  ---
  Executive Summary

  - Readiness: Close
  - RACE findings:  1 — one confirmed use-after-realloc race in the buffer protocol
  - UNSAFE findings: 0
  - PROTECT findings: 0
  - MIGRATE findings: 1

  The C extension is in strong shape. Commit af45640 correctly applied
  @cython.critical_section to all 17 public Python-visible methods across
  Packer and Unpacker. The only gap is two buffer-protocol slots
  (__getbuffer__ and __releasebuffer__) that were missed in the same sweep.
  Without that decorator, any thread calling memoryview(packer) concurrently
  with pack() can trigger a use-after-realloc on the raw pk.buf pointer.

  Verified fix: @cython.critical_section compiles on both __getbuffer__ and
  __releasebuffer__ in Cython 3.2.5 — the fix is two decorator lines.

  Two tensions are worth explicit attention: (1) freethreading_compatible = True
  is declared on the C extension module only; the pure-Python fallback
  (msgpack/fallback.py) carries zero synchronization, is the silent drop-in
  on PyPy and under MSGPACK_PUREPYTHON=1, and has no thread-safety
  documentation. (2) The CI stress test that should validate this is never
  collected by pytest due to a filename typo, weakening the "FT tests pass"
  signal.

  ---
  Findings by Priority

  ───────────────────────────────────────────────────────────────────────────────
  RACE Findings (fix immediately) — 1
  ───────────────────────────────────────────────────────────────────────────────

  #: 1
  Finding: Packer.__getbuffer__ and __releasebuffer__ missing
    @cython.critical_section — two independent races:

    Race A (memory safety): __getbuffer__ reads self.pk.buf and self.pk.length
    via PyBuffer_FillInfo, then increments self.exports — all without holding
    the per-object lock. Concurrently, pack() holds that lock, reads
    self.exports via _check_exports(), and if it sees exports == 0 (possible
    due to Race B below), calls msgpack_pack_write → PyMem_Realloc(pk.buf).
    The consumer of the Py_buffer now holds a pointer to freed-or-moved memory.

    Race B (counter integrity): self.exports is a cdef size_t that is
    incremented in __getbuffer__ and decremented in __releasebuffer__ without
    any lock or atomic operation. Two concurrent memoryview(packer) calls or
    concurrent releases race on a plain read-modify-write. A lost update
    makes the counter negative or zero, defeating the _check_exports() guard
    that serializes pack/reset against live buffer exports.

    Sub-note: the public Packer.getbuffer() method (line 356) also lacks
    @cython.critical_section. It calls memoryview(self) which triggers the
    slot directly. Decorating getbuffer() alone is not sufficient (any caller
    of memoryview() bypasses it), but it was missed in the same af45640 sweep.

  File:Line: _packer.pyx:363-368 (slots); _packer.pyx:356 (getbuffer())
  Severity: CRITICAL
  Agents: shared-state-auditor, ft-history-analyzer, lock-discipline-checker,
          atomic-candidate-finder, unsafe-api-detector (5-agent cross-validation)

  Fix (verified to compile in Cython 3.2.5):

    # _packer.pyx:362-368 — add two decorators
    @cython.critical_section
    def __getbuffer__(self, Py_buffer *buffer, int flags):
        PyBuffer_FillInfo(buffer, self, self.pk.buf, self.pk.length, 1, flags)
        self.exports += 1

    @cython.critical_section
    def __releasebuffer__(self, Py_buffer *buffer):
        self.exports -= 1

    # _packer.pyx:355 — add decorator to public method as defence-in-depth
    @cython.critical_section
    def getbuffer(self):
        return memoryview(self)

  Note: the decorator emits Py_BEGIN_CRITICAL_SECTION(self) /
  Py_END_CRITICAL_SECTION() in the generated C, using the same per-object
  mutex that pack(), reset(), etc. already hold. This makes the
  exports-check-then-realloc in pack() atomic with respect to
  buffer-export-then-increment in __getbuffer__. The lock will contend with
  pack() — this is correct and intentional.

  ───────────────────────────────────────────────────────────────────────────────
  MIGRATE Findings (structural changes) — 1
  ───────────────────────────────────────────────────────────────────────────────

  #: 2
  Finding: PyUnicode_InternInPlace called on every map key from untrusted
    msgpack data (unpack.h:191-193). The API is deprecated since Python 3.12
    in favour of PyUnicode_InternImmortal. Under free-threading CPython 3.13+
    the intern table is internally locked so there is no undefined behavior,
    but the lock contention is proportional to key throughput across all
    concurrent unpackers. The documented replacement is available at Python
    ≥3.10 (this project's minimum target).

  File:Line: unpack.h:191-193
  Severity: MEDIUM
  Agent: unsafe-api-detector

  Fix:
    // unpack.h:191-193 — replace
    if (PyUnicode_CheckExact(k)) {
        PyUnicode_InternImmortal(&k);   // was: PyUnicode_InternInPlace(&k)
    }

  Note: PyUnicode_InternImmortal is explicitly documented as thread-safe and
  avoids the intern-table lock contention under heavy parallel unpack load.
  It marks strings as immortal, which has a slight memory-use implication
  but matches the observed usage here (map keys are typically short-lived
  keys not intended to be GC'd individually anyway).

  ---
  Tensions

  Tension 1: freethreading_compatible on C extension, but not fallback.py

  freethreading_compatible = True is declared on the C extension module only.
  msgpack/fallback.py — the silent drop-in on PyPy and under
  MSGPACK_PUREPYTHON=1 — uses no threading primitives at all. No Lock, no
  RLock, no synchronization around any mutable Packer or Unpacker instance
  state. The CI addition in 9de2fd9 does run fallback.py under
  --parallel-threads=auto --iterations=20, but that test harness creates a
  separate instance per test function (it parallelises tests, not shared-
  object calls), so it does not exercise shared-instance races.

  A user on PyPy who sees freethreading_compatible = True (either from the
  wheel metadata or from msgpack's documentation) has no indication that the
  fallback they are running is unprotected.

  Options:
    A. Add an explicit class-level or module-level note to fallback.py
       documenting "not thread-safe when instances are shared across threads."
    B. Add threading.RLock protection to Packer and Unpacker in fallback.py,
       mirroring the @cython.critical_section pattern from the C extension.
    C. Raise an explicit RuntimeError or warning when fallback.py is loaded
       under a free-threaded Python build (PYTHON_GIL=0 / sys._is_gil_enabled()
       returns False).

  Tension 2: CI stress test never runs

  test/uneeded_test_multithreading.py contains a genuine multi-threaded
  stress test for Packer.pack under concurrent threads. It is never collected
  by pytest because its filename does not match pytest's default discovery
  patterns (test_*.py / *_test.py). The "uneeded" typo is also present in
  the filename itself.

  The CI run in 9de2fd9 therefore validates that parallel *test functions*
  (each with their own Packer/Unpacker) pass — not that a single shared
  Packer instance is safe under concurrent writes.

  Fix: rename to test_multithreading.py and add a pytest mark so it is
  included in the FT CI matrix but skipped on non-FT builds:

    # test_multithreading.py
    import sys, pytest
    pytestmark = pytest.mark.skipif(
        sys.version_info < (3, 13) or getattr(sys, "_is_gil_enabled", lambda: True)(),
        reason="free-threaded Python only"
    )

  ---
  SAFE Patterns (confirmed thread-safe)

  - Per-instance Packer state (pk.buf, pk.length, pk.buf_size, use_bin_type,
    _default, strict_types, use_float, autoreset, datetime):
    All 8 mutating methods carry @cython.critical_section. No cdef helper
    acquires the lock; helpers are called only from within decorated callers.
    SAFE — lock-discipline-checker confirmed zero mismatches across 17 methods.

  - Per-instance Unpacker state (ctx, buf, buf_head, buf_tail, stream_offset,
    file_like, all hook references):
    All 9 public methods carry @cython.critical_section.
    SAFE — lock-discipline-checker: all callers confirmed, no missed paths.

  - stream_offset: written only from decorated methods (unpack, skip,
    read_array_header, read_map_header, __next__, read_bytes).
    SAFE — atomic-candidate-finder confirmed.

  - Module-level globals (utc, epoch, giga, ExtType, Timestamp,
    DEFAULT_RECURSE_LIMIT, ITEM_LIMIT):
    Written exactly once during module exec, which runs under CPython's
    import lock before any user thread can access the module. Thereafter
    read-only. SAFE under free-threading.
    (Subinterpreter isolation concern — these are process-global PyObject*
    statics; see cext-analysis.txt findings 8–10 for that separate issue.)

  - Hand-written C headers (pack.h, pack_template.h, unpack.h,
    unpack_template.h, unpack_container_header.h, unpack_define.h, sysdep.h):
    Zero file-scope static variables. All mutable state flows through
    struct arguments (msgpack_packer *, unpack_context *). SAFE.

  - unpackb() function: allocates its unpack_context on the stack.
    No shared state. Fully thread-safe as a standalone function.
    SAFE.

  - No GIL-released regions: zero Py_BEGIN_ALLOW_THREADS, zero
    with nogil:, zero PyGILState_Ensure/Release in the entire codebase.
    The CRITICAL category "unsafe API outside GIL" has zero instances.

  - No StopTheWorld needed: module init serialized by import lock;
    GC traversal handled by CPython's own STW before any gc.collect();
    no _PyEval_StopTheWorld usage added or needed.

  ---
  Recommended Action Plan

  Immediate (RACE)

  1. Fix Finding 1 — Add @cython.critical_section to __getbuffer__,
     __releasebuffer__, and getbuffer() in _packer.pyx:355-368. Three
     one-line additions. Verified to compile in Cython 3.2.5. Run make cython
     after editing to regenerate _cmsgpack.c, then rebuild and run:
       pytest -v test/ --parallel-threads=auto --iterations=20
     to confirm no race is detectable.

  Immediate (CI infrastructure)

  2. Rename test/uneeded_test_multithreading.py → test/test_multithreading.py
     and add the skipif mark shown above. This ensures the concurrent-pack
     stress test actually runs in the CI matrix that 9de2fd9 added.

  Short-term (MIGRATE)

  3. Fix Finding 2 — Replace PyUnicode_InternInPlace with
     PyUnicode_InternImmortal in unpack.h:191-193. One-line change.

  Longer-term (Tensions)

  4. Decide and document the fallback.py thread-safety posture. Either add
     threading.RLock protection (mirroring the C extension's pattern) or add
     explicit "not thread-safe for shared instances" documentation and a
     runtime warning when loaded under a GIL-disabled interpreter.

  5. Once the C extension subinterpreter migration is completed (see
     cext-analysis.txt action 14 — CYTHON_USE_MODULE_STATE=1), the
     freethreading_compatible + multiple_interpreters story becomes fully
     consistent.

  ---
  Cross-reference

  The subinterpreter safety findings (module-level PyObject* globals,
  PyDateTimeAPI cross-interpreter, Py_mod_multiple_interpreters slot)
  are covered separately in cext-analysis.txt findings 8-10 and P3.
  They are FT-safe and are not repeated here.