Stopping the world means waiting for every running thread to reach a safepoint.
A thread blocked acquiring a lock reaches none, so a thread that waits for a
lock while attached is a thread the world cannot stop for as long as it waits.

What changed

RawDetachingRwLock wraps the raw rwlock and hands the wait for a contended
acquire to a hook that leaves the interpreter first. An acquire that takes the
lock on its first try is the same atomic exchange it was. The hook lives in the
vm, since rustpython-common cannot depend on it, and runs allow_threads;
initialize_vm installs it, idempotently, so every interpreter in a process can
call it.

Nothing spins before the wait. parking_lot already spins before it parks and
already skips that spin once a waiter has parked — state & (PARKED_BIT | WRITER_PARKED_BIT) == 0 && spinwait.spin(), the same condition
_PyMutex_LockTimed spins under. A spin layered on top cannot read that
condition, and would go on retrying a try_lock that reports failure for as
long as a writer holds WRITER_BIT, which it takes before it waits for readers
to drain. An earlier revision of this PR had one; measured against bytearray
contention (reader/writer mixes, short and long holds, alternating binaries)
it was a wash in both directions, so it is gone.

PyByteArray::inner is the first user. BorrowedValue/BorrowedValueMut gain
the matching mapped-guard variants.

Only lock_shared and lock_exclusive detach. upgrade runs with the
upgradable lock already held and lock_shared_recursive may be the re-entrant
take of a lock this thread holds; detaching there parks a thread holding the
lock. lock_upgradable starts from holding nothing and could detach safely, but
nothing takes an upgradable read of one of these, so it does not.

Why this is opt-in

The wait ends with the lock acquired while detached, so the thread comes back
holding it — and re-attaching is a point at which a stop-the-world in flight
parks it. Everything that stops the world must therefore be able to finish
without that lock, so the rule for opting a lock in is that nothing reachable
from a stop-the-world section takes it.

Not implementing Traverse for PyDetachingRwLock states that to a
collection: a payload holding one cannot derive Traverse, so a collection
cannot walk into it. It says nothing to the other sections — dumping
tracebacks, enumerating thread frames, forking — and #[pytraverse(skip)]
steps around it besides.

So the rule is also an assertion. set_world_stopped records, on the one
thread still running inside a stopped world, that it is that thread, and the
blocking acquires assert it is not set. Debug builds only. Nothing in the tree
trips it: 180 rounds of collect, sys._current_frames,
faulthandler.dump_traceback and 18 forks with four threads churning
bytearrays, all clean.

Also here: the sites that still waited badly

os.readinto held the destination's write lock for the whole call, including
the read(2) inside allow_threads. On a pipe, socket or terminal that read
returns only when the other end writes, so another thread reaching the same
object waited on that lock for an unbounded time. It now reads into scratch and
takes the lock only for the copy, unless the fd answers without waiting — which
is what FileIO.readinto, socket.recv_into and socket.recvfrom_into
already do. os.readinto was the site left over.

fcntl ran every one of its calls with the thread attached, so a thread
inside one reached no safepoint until it returned — and flock(LOCK_EX) and
lockf(F_LOCK) return when whoever holds the lock gives it up, which may be
never. fcntl_fcntl_impl, fcntl_ioctl_impl, fcntl_flock_impl and
fcntl_lockf_impl all release around the call; these now do too. ioctl with
mutate_flag also held the target's write lock for the whole call, and now
copies through a buffer of its own, as fcntl_ioctl_impl does for anything up
to IOCTL_BUFSZ.

SSLSocket.read on the openssl backend wrote straight into the destination,
holding its lock until the peer answered. It reads aside and takes the lock for
the copy, which is what the rustls backend already does.

The openssl backend's SSL calls ran attached. With a timeout set the wait
lands in select → sock_wait, which already detaches; with settimeout(None)
there is no such return and SSL_read blocks in recv(2) through
impl Read for &PySocket, holding the connection's write lock. A server that
finished a handshake and then said nothing deadlocked the whole process — the
collector suspended the main thread at a safepoint and then waited forever for
the reader, so even a Python-level timeout could not fire. SSL_do_handshake,
SSL_read_ex, SSL_write_ex and SSL_shutdown now detach as they do in
_ssl.c, and the connection lock becomes a detaching one since it stays held
across the call. That collection now completes in 3 ms.

Detaching there means the SSL callbacks — _servername_callback,
_msg_callback — would run Python on a thread the requester counts as parked,
so they attach first. _servername_callback opens with PyGILState_Ensure()
for the same reason. The vm had no inverse of allow_threads to mirror it
with, so attach_for_callback is new: it attaches for the closure and returns
the thread to where it was, and routes a thread already moved to SUSPENDED
through attach_thread so it parks until the world starts again.

That is worth being explicit about, because it narrows what this PR is for. The
earlier _queue/_thread/_io/_winapi work already closed the holders
that kept an object lock across a blocking call, and os.readinto was the last
one; FileIO.write and socket.send* copy through borrow_buf_unlocked,
FileIO.readinto and socket.recv_into read into scratch. So what remains for
the detaching lock is waiters blocked behind a bounded hold, not the unbounded
holds the holder fixes removed.

Tests

a_thread_blocked_on_a_lock_does_not_stall_stop_the_world holds a
PyDetachingRwLock, blocks an interpreter thread on it, and asserts
stop-the-world still completes. It runs the stop on a thread of its own with a
timeout, so a stop that never completes fails the test rather than hanging it.
With the hook installation commented out it fails on the 10 s timeout; with it,
it passes in 0.07 s.

a_callback_inside_a_detached_call_waits_for_the_world stops the world with a
thread detached, turns that thread loose at a callback and asserts it does not
run until the world starts again. With the transition disabled it fails.

taking_one_while_stopping_the_world_is_caught takes a detaching lock with the
stopped-world flag set and asserts the panic, so that guard is not dead code.

Run locally: the full workspace test command, CI clippy for the rustls and
openssl feature sets, cargo doc (no new warnings), a bytearray/memoryview
stress across 8 threads with 2 concurrent gc.collect() loops, os.readinto
checked against CPython on a regular file, a pipe, a short buffer and a
memoryview target, and test_fcntl test_ioctl test_os test_posix test_fileio test_bytes test_memoryview test_threading test_io test_gc test_buffer test_ssl test_asyncio (46/46 files, 4,986 tests, on the rustls backend). Every commit
was checked to build on its own.

The openssl backend is not built in CI. Built here: test_ssl fails the same 8
tests before and after, by name; openssl.rs draws no clippy warning it did not
draw before; 40 handshakes with a Python _msg_callback firing from inside
them under a looping collector, 920 callback invocations, five runs clean.

Not done here

_PyRWMutex acquires after re-attaching — rwmutex_set_parked_and_wait
parks detached, and the retry loop in _PyRWMutex_RLock takes the lock once the
thread is back — so no thread is ever parked holding one. Mirroring that would
mean a raw rwlock built on parking_lot_core rather than wrapping
parking_lot, and it would not lift the opt-in rule anyway: a holder inside
allow_threads is parked holding the lock too, and that path is untouched by
how waiters acquire.

Closing that path is what _PyCriticalSection_SuspendAll does: detach_thread
unlocks every critical section the thread holds and attaching resumes them, so
no object lock is ever held across a detached window and no opt-in rule is
needed. That cannot be mirrored here. These locks hand out borrow-checked
references — a guard derefs to &T/&mut T — so releasing the raw lock while
a guard is alive would let another thread produce an aliasing &mut T.
Py_BEGIN_CRITICAL_SECTION hands out no borrow and requires state to be
re-read after resume, which is what buys CPython the freedom. The Rust form of
the same invariant is not holding the guard across the call, which is what
borrow_buf_unlocked, the scratch copies above and os.readinto do, with the
detaching lock for the places where it genuinely must be held.

Neither ssl nor openssl calls allow_threads, but only one of them needs
to. The rustls backend performs no blocking I/O of its own: it reaches the
socket through the Python socket methods, and sock_io and
sock_wait_deadline already detach around the syscall and the poll. What
rustls itself does — read_tls from a Cursor, process_new_packets — is
in-memory work.

The openssl backend does block attached, but only when the socket is in
blocking mode. With a timeout set, PySocket is O_NONBLOCK, SSL_read
returns WANT_READ, and the wait lands in select → sock_wait, detached.
With settimeout(None) there is no such return: SSL_read blocks in recv(2)
through impl Read for &PySocket, attached, holding self.connection.write().
Fixing it is not a wrap, because _servername_callback and _msg_callback run
Python from inside the SSL call — _servername_callback opens with
PyGILState_Ensure() for exactly that reason, and this vm has no public
inverse of allow_threads to mirror it with. That primitive comes first.

Summary by CodeRabbit

• New Features
  • Blocking file-control, I/O, and OpenSSL operations now release the interpreter while waiting, improving responsiveness in threaded applications.
  • Python callbacks invoked during detached operations are handled safely and resume with the correct interpreter context.
  • Bytearray and buffer operations now support improved concurrent access.
• Bug Fixes
  • Improved handling of blocking reads and mutable buffers, including safer data copying during system calls.
  • Thread coordination now remains reliable during stop-the-world pauses and process-fork resets.