FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-130115: fix thread identifiers for 32-bit musl by vfazio · Pull Request #130391 · python/cpython · GitHub

/ cpython Public

gh-130115: fix thread identifiers for 32-bit musl - #130391

Merged
pitrou merged 11 commits into
python:mainfrom
vfazio:vfazio-thread_get_ident
Apr 4, 2025
ConversationCommits (11)ChecksFiles changed
Merged

gh-130115: fix thread identifiers for 32-bit musl#130391
pitrou merged 11 commits into
python:mainfrom
vfazio:vfazio-thread_get_ident

Conversation

vfazio commented Feb 21, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

CPython's pthread-based thread identifier relies on pthread_t being able to be represented as an unsigned integer type.

This is true in most Linux libc implementations where it's defined as an unsigned long, however musl typedefs it as a struct *.

If the pointer has the high bit set and is cast to PyThread_ident_t, the resultant value can be sign-extended (https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Arrays-and-pointers-implementation.html). This can cause issues when comparing against threading._MainThread's identifier. The main thread's identifier value is retrieved via _get_main_thread_ident which is backed by an unsigned long which truncates sign extended bits.

  >>> hex(threading.main_thread().ident)
  '0xb6f33f3c'
  >>> hex(threading.current_thread().ident)
  '0xffffffffb6f33f3c'

Work around this by making a new function which translates a pthread_t to PyThread_ident_t by casting through an integer type of the appropriate size to avoid sign extension. Factoring this out allows us to replace the internal logic in the future with some min-heap or other data structure to manage pthread_t objects in a more opaque fashion.

Note musl isn't "officially" supported in PEP 11, however platform detection was added in c163d7f and similar PRs have been merged in the past which target it 5633c4f

This PR is intended to be a "minimum" to get this working. Longer term there should maybe be work to keep pthread_t opaque and not make assumptions about its type.

Return the main thread's short identifier on certain platforms
51d6579 
CPython's pthread-based thread identifier relies on pthread_t being able
to be represented as an unsigned integer type.

This is true in most Linux libc implementations where it's defined as an
unsigned long, however musl typedefs it as a struct *.

If the pointer has the high bit set and is cast to PyThread_ident_t, the
resultant value can be sign-extended [0]. This can cause issues when
comparing against threading._MainThread's identifier. The main thread's
identifier value is retrieved via _get_main_thread_ident which is backed
by an unsigned long which truncates sign extended bits.

  >>> hex(threading.main_thread().ident)
  '0xb6f33f3c'
  >>> hex(threading.current_thread().ident)
  '0xffffffffb6f33f3c'

Work around this by conditionally compiling in some code for non-glibc
based Linux platforms that are at risk of sign-extension to return a
PyLong based on the main thread's unsigned long thread identifier if the
current thread is the main thread.

[0]: https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Arrays-and-pointers-implementation.html

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Add NEWS entry
a9ca268 
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
bedevere-app Bot mentioned this pull request Feb 21, 2025
Closed
vfazio marked this pull request as draft February 22, 2025 16:59

vfazio commented Feb 22, 2025

Copy link
Copy Markdown
Contributor Author

Drafting this.

After doing some initial testing, changing only get_ident is not sufficient for the threading unit tests to pass. There are numerous edge cases to account for, namely things like:

  • test_2_join_in_forked_process requiring ThreadHandle.join() to work
  • test_excepthook_thread_None requiring the thread identifier in the exception string to match
  • test_main_thread_after_fork_from_foreign_thread(dummy=True) converting a dummy thread to a main thread testing that the identifiers are the same
  • PyThread_start_joinable_thread creates its own identifier and there's expectation's that the handle's identifier will be consistent
    (test_foreign_thread, test_reinit_tls_after_fork, test_ident_of_no_threading_threads, test_main_thread_after_fork_from_nonmain_thread)

I plan on testing on actual hardware (RPi on 32bit Alpine) at some point, but I've emulated this environment in Docker using arm32v7/alpine & the qemu user space emulator in the mean time.

I'll rework the patch and then validate on hardware as well

Merge remote-tracking branch 'upstream/main' into vfazio-thread_get_i…
b93868e 
…dent
Move pthread_t to ident conversion to a shared function
7a4de5f 
Move this to a static function so all callers have consistent behavior.

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
vfazio changed the title gh-130115: fix return value of threading.get_ident for the main thread on 32bit musl gh-130115: fix thread identifiers for 32-bit musl Feb 23, 2025
Reword NEWS entry
af1233b 
Signed-off-by: Vincent Fazio <vfazio@gmail.com>

vfazio commented Feb 23, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I built python --with-assertions for warm and fuzzies and ran the threading tests w/o issue:

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.21.3
PRETTY_NAME="Alpine Linux v3.21"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# /lib/ld-musl-armhf.so.1
musl libc (armhf)
Version 1.2.5
Dynamic Program Loader
Usage: /lib/ld-musl-armhf.so.1 [options] [--] pathname [args]

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# git rev-parse HEAD
af1233b283e1fd6343dad8a2a2605a8009bb4556

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# ./python -m test -v test_threading
== CPython 3.14.0a5+ (heads/vfazio-thread_get_ident:af1233b283, Feb 23 2025, 02:09:24) [GCC 14.2.0]
== Linux-6.8.0-51-generic-armv7l-with little-endian
== Python build: release with_assert
== cwd: /tmp/tmp.pdpMFD/cpython/build/test_python_worker_176382æ
== CPU count: 32
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3615652272
0:00:00 load avg: 0.15 Run 1 test sequentially in a single process
0:00:00 load avg: 0.15 [1/1] test_threading
test_atexit_after_shutdown (test.test_threading.AtexitTests.test_atexit_after_shutdown) ... ok
test_atexit_called_once (test.test_threading.AtexitTests.test_atexit_called_once) ... ok
test_atexit_output (test.test_threading.AtexitTests.test_atexit_output) ... ok
test_abort (test.test_threading.BarrierTests.test_abort)
Test that an abort will put the barrier in a broken state ... ok
test_abort_and_reset (test.test_threading.BarrierTests.test_abort_and_reset)
Test that a barrier can be reset after being broken. ... ok
test_action (test.test_threading.BarrierTests.test_action)
Test the 'action' callback ... ok
test_barrier (test.test_threading.BarrierTests.test_barrier)
Test that a barrier is passed in lockstep ... ok
test_barrier_10 (test.test_threading.BarrierTests.test_barrier_10)
Test that a barrier works for 10 consecutive runs ... ok
test_constructor (test.test_threading.BarrierTests.test_constructor) ... ok
test_default_timeout (test.test_threading.BarrierTests.test_default_timeout)
Test the barrier's default timeout ... ok
test_repr (test.test_threading.BarrierTests.test_repr) ... ok
test_reset (test.test_threading.BarrierTests.test_reset)
Test that a 'reset' on a barrier frees the waiting threads ... ok
test_single_thread (test.test_threading.BarrierTests.test_single_thread) ... ok
test_timeout (test.test_threading.BarrierTests.test_timeout)
Test wait(timeout) ... ok
test_wait_return (test.test_threading.BarrierTests.test_wait_return)
test the return value from barrier.wait ... ok
test_acquire (test.test_threading.BoundedSemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.BoundedSemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.BoundedSemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.BoundedSemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.BoundedSemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.BoundedSemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.BoundedSemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.BoundedSemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.BoundedSemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.BoundedSemaphoreTests.test_with) ... ok
test__is_owned (test.test_threading.CRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.CRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.CRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.CRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.CRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.CRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.CRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.CRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.CRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.CRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.CRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.CRLockTests.test_repr) ... ok
test_signature (test.test_threading.CRLockTests.test_signature) ... ok
test_thread_leak (test.test_threading.CRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.CRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.CRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.CRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.CRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.CRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.CRLockTests.test_with) ... ok
test__is_owned (test.test_threading.ConditionAsRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.ConditionAsRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.ConditionAsRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.ConditionAsRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.ConditionAsRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.ConditionAsRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.ConditionAsRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.ConditionAsRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.ConditionAsRLockTests.test_recursion_count) ... skipped 'Condition does not expose _recursion_count()'
test_release_save_unacquired (test.test_threading.ConditionAsRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.ConditionAsRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.ConditionAsRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.ConditionAsRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.ConditionAsRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.ConditionAsRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.ConditionAsRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.ConditionAsRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.ConditionAsRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.ConditionAsRLockTests.test_with) ... ok
test_acquire (test.test_threading.ConditionTests.test_acquire) ... ok
test_notify (test.test_threading.ConditionTests.test_notify) ... ok
test_timeout (test.test_threading.ConditionTests.test_timeout) ... ok
test_unacquired_notify (test.test_threading.ConditionTests.test_unacquired_notify) ... ok
test_unacquired_wait (test.test_threading.ConditionTests.test_unacquired_wait) ... ok
test_waitfor (test.test_threading.ConditionTests.test_waitfor) ... ok
test_waitfor_timeout (test.test_threading.ConditionTests.test_waitfor_timeout) ... ok
test_at_fork_reinit (test.test_threading.EventTests.test_at_fork_reinit) ... ok
test_is_set (test.test_threading.EventTests.test_is_set) ... ok
test_notify (test.test_threading.EventTests.test_notify) ... ok
test_repr (test.test_threading.EventTests.test_repr) ... ok
test_set_and_clear (test.test_threading.EventTests.test_set_and_clear) ... ok
test_timeout (test.test_threading.EventTests.test_timeout) ... ok
test_custom_excepthook (test.test_threading.ExceptHookTests.test_custom_excepthook) ... ok
test_custom_excepthook_fail (test.test_threading.ExceptHookTests.test_custom_excepthook_fail) ... ok
test_excepthook (test.test_threading.ExceptHookTests.test_excepthook) ... ok
test_excepthook_thread_None (test.test_threading.ExceptHookTests.test_excepthook_thread_None) ... ok
test_original_excepthook (test.test_threading.ExceptHookTests.test_original_excepthook) ... ok
test_system_exit (test.test_threading.ExceptHookTests.test_system_exit) ... ok
test_can_interrupt_tight_loops (test.test_threading.InterruptMainTests.test_can_interrupt_tight_loops) ... ok
test_interrupt_main_invalid_signal (test.test_threading.InterruptMainTests.test_interrupt_main_invalid_signal) ... ok
test_interrupt_main_mainthread (test.test_threading.InterruptMainTests.test_interrupt_main_mainthread) ... ok
test_interrupt_main_noerror (test.test_threading.InterruptMainTests.test_interrupt_main_noerror) ... ok
test_interrupt_main_subthread (test.test_threading.InterruptMainTests.test_interrupt_main_subthread) ... ok
test_interrupt_main_with_signal_handler (test.test_threading.InterruptMainTests.test_interrupt_main_with_signal_handler) ... ok
test_acquire_contended (test.test_threading.LockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.LockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.LockTests.test_acquire_release) ... ok
test_at_fork_reinit (test.test_threading.LockTests.test_at_fork_reinit) ... ok
test_constructor (test.test_threading.LockTests.test_constructor) ... ok
test_different_thread (test.test_threading.LockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.LockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.LockTests.test_reacquire) ... ok
test_repr (test.test_threading.LockTests.test_repr) ... ok
test_state_after_timeout (test.test_threading.LockTests.test_state_after_timeout) ... ok
test_thread_leak (test.test_threading.LockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.LockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.LockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.LockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.LockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.LockTests.test_weakref_exists) ... ok
test_with (test.test_threading.LockTests.test_with) ... ok
test__all__ (test.test_threading.MiscTestCase.test__all__) ... ok
test_change_name (test.test_threading.MiscTestCase.test_change_name) ... ok
test_set_name (test.test_threading.MiscTestCase.test_set_name) ... ok
test__is_owned (test.test_threading.PyRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.PyRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.PyRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.PyRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.PyRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.PyRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.PyRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.PyRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.PyRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.PyRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.PyRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.PyRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.PyRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.PyRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.PyRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.PyRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.PyRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.PyRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.PyRLockTests.test_with) ... ok
test_acquire (test.test_threading.SemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.SemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.SemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.SemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.SemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.SemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.SemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.SemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.SemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.SemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.SemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.SemaphoreTests.test_with) ... ok
test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadingTests.test_daemon_threads_fatal_error) ... ok
test_daemon_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_daemon_threads_not_allowed) ... ok
test_threads_join (test.test_threading.SubinterpThreadingTests.test_threads_join) ... ok
test_threads_join_2 (test.test_threading.SubinterpThreadingTests.test_threads_join_2) ... ok
test_threads_join_with_no_main (test.test_threading.SubinterpThreadingTests.test_threads_join_with_no_main) ... ok
test_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_threads_not_allowed) ... ok
test_1_join_on_shutdown (test.test_threading.ThreadJoinOnShutdown.test_1_join_on_shutdown) ... ok
test_2_join_in_forked_process (test.test_threading.ThreadJoinOnShutdown.test_2_join_in_forked_process) ... ok
test_3_join_in_forked_from_thread (test.test_threading.ThreadJoinOnShutdown.test_3_join_in_forked_from_thread) ... ok
test_4_daemon_threads (test.test_threading.ThreadJoinOnShutdown.test_4_daemon_threads) ... ok
test_clear_threads_states_after_fork (test.test_threading.ThreadJoinOnShutdown.test_clear_threads_states_after_fork) ... ok
test_reinit_tls_after_fork (test.test_threading.ThreadJoinOnShutdown.test_reinit_tls_after_fork) ... ok
test_thread_from_thread (test.test_threading.ThreadJoinOnShutdown.test_thread_from_thread) ... ok
test_BoundedSemaphore_limit (test.test_threading.ThreadTests.test_BoundedSemaphore_limit) ... ok
test_PyThreadState_SetAsyncExc (test.test_threading.ThreadTests.test_PyThreadState_SetAsyncExc) ... skipped "No module named '_ctypes'"
test_args_argument (test.test_threading.ThreadTests.test_args_argument) ... ok
test_boolean_target (test.test_threading.ThreadTests.test_boolean_target) ... ok
test_daemon_param (test.test_threading.ThreadTests.test_daemon_param) ... ok
test_dummy_thread_after_fork (test.test_threading.ThreadTests.test_dummy_thread_after_fork) ... ok
test_enumerate_after_join (test.test_threading.ThreadTests.test_enumerate_after_join) ... ok
test_finalization_shutdown (test.test_threading.ThreadTests.test_finalization_shutdown) ... ok
test_finalize_daemon_thread_hang (test.test_threading.ThreadTests.test_finalize_daemon_thread_hang) ... ok
test_finalize_running_thread (test.test_threading.ThreadTests.test_finalize_running_thread) ... skipped "No module named '_ctypes'"
test_finalize_with_trace (test.test_threading.ThreadTests.test_finalize_with_trace) ... ok
test_foreign_thread (test.test_threading.ThreadTests.test_foreign_thread) ... ok
test_frame_tstate_tracing (test.test_threading.ThreadTests.test_frame_tstate_tracing) ... ok
test_getprofile (test.test_threading.ThreadTests.test_getprofile) ... ok
test_getprofile_all_threads (test.test_threading.ThreadTests.test_getprofile_all_threads) ... ok
test_gettrace (test.test_threading.ThreadTests.test_gettrace) ... ok
test_gettrace_all_threads (test.test_threading.ThreadTests.test_gettrace_all_threads) ... ok
test_ident_of_no_threading_threads (test.test_threading.ThreadTests.test_ident_of_no_threading_threads) ... ok
test_import_from_another_thread (test.test_threading.ThreadTests.test_import_from_another_thread) ... ok
test_is_alive_after_fork (test.test_threading.ThreadTests.test_is_alive_after_fork) ... ok
test_join_from_multiple_threads (test.test_threading.ThreadTests.test_join_from_multiple_threads) ... ok
test_join_nondaemon_on_shutdown (test.test_threading.ThreadTests.test_join_nondaemon_on_shutdown) ... ok
test_join_with_timeout (test.test_threading.ThreadTests.test_join_with_timeout) ... ok
test_leak_without_join (test.test_threading.ThreadTests.test_leak_without_join) ... ok
test_limbo_cleanup (test.test_threading.ThreadTests.test_limbo_cleanup) ... ok
test_locals_at_exit (test.test_threading.ThreadTests.test_locals_at_exit) ... ok
test_lock_no_args (test.test_threading.ThreadTests.test_lock_no_args) ... ok
test_lock_no_subclass (test.test_threading.ThreadTests.test_lock_no_subclass) ... ok
test_lock_or_none (test.test_threading.ThreadTests.test_lock_or_none) ... ok
test_main_thread (test.test_threading.ThreadTests.test_main_thread) ... ok
test_main_thread_after_fork (test.test_threading.ThreadTests.test_main_thread_after_fork) ... ok
test_main_thread_after_fork_from_dummy_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_dummy_thread) ... ok
test_main_thread_after_fork_from_foreign_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_foreign_thread) ... ok
test_main_thread_after_fork_from_nonmain_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_nonmain_thread) ... ok
test_main_thread_during_shutdown (test.test_threading.ThreadTests.test_main_thread_during_shutdown) ... ok
test_name (test.test_threading.ThreadTests.test_name) ... ok
test_no_refcycle_through_target (test.test_threading.ThreadTests.test_no_refcycle_through_target) ... ok
test_old_threading_api (test.test_threading.ThreadTests.test_old_threading_api) ... ok
test_repr_daemon (test.test_threading.ThreadTests.test_repr_daemon) ... ok
test_repr_stopped (test.test_threading.ThreadTests.test_repr_stopped) ... ok
test_start_new_thread_at_finalization (test.test_threading.ThreadTests.test_start_new_thread_at_finalization) ... ok
test_start_new_thread_failed (test.test_threading.ThreadTests.test_start_new_thread_failed) ... skipped 'RLIMIT_NPROC had no effect; probably superuser'
test_various_ops (test.test_threading.ThreadTests.test_various_ops) ... task <thread 0> will run for 72.2 usec
1 tasks are running
task <thread 1> will run for 98.7 usec
task <thread 0> done
task <thread 2> will run for 61.4 usec
task <thread 3> will run for 52.4 usec
task <thread 4> will run for 44.5 usec
task <thread 5> will run for 14.9 usec
task <thread 6> will run for 66.8 usec
task <thread 7> will run for 45.8 usec
task <thread 8> will run for 48.9 usec
task <thread 9> will run for 94.0 usec
waiting for all tasks to complete
2 tasks are running
<thread 0> is finished. 1 tasks are running
task <thread 1> done
2 tasks are running
3 tasks are running
<thread 1> is finished. 2 tasks are running
task <thread 3> done
<thread 3> is finished. 1 tasks are running
2 tasks are running
task <thread 2> done
<thread 2> is finished. 1 tasks are running
2 tasks are running
task <thread 4> done
3 tasks are running
task <thread 5> done
<thread 4> is finished. 2 tasks are running
<thread 5> is finished. 1 tasks are running
task <thread 6> done
2 tasks are running
3 tasks are running
<thread 6> is finished. 2 tasks are running
task <thread 7> done
<thread 7> is finished. 1 tasks are running
task <thread 8> done
2 tasks are running
<thread 8> is finished. 1 tasks are running
task <thread 9> done
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_large_stack (test.test_threading.ThreadTests.test_various_ops_large_stack) ... with 1 MiB thread stack size...
task <thread 0> will run for 93.7 usec
1 tasks are running
task <thread 1> will run for 76.0 usec
task <thread 0> done
task <thread 2> will run for 50.7 usec
task <thread 3> will run for 92.3 usec
task <thread 4> will run for 11.0 usec
task <thread 5> will run for 9.0 usec
2 tasks are running
task <thread 6> will run for 2.9 usec
<thread 0> is finished. 1 tasks are running
task <thread 8> will run for 48.2 usec
task <thread 9> will run for 27.5 usec
waiting for all tasks to complete
2 tasks are running
task <thread 7> will run for 86.2 usec
task <thread 1> done
3 tasks are running
<thread 1> is finished. 2 tasks are running
task <thread 2> done
<thread 2> is finished. 1 tasks are running
task <thread 3> done
2 tasks are running
3 tasks are running
task <thread 5> done
<thread 3> is finished. 2 tasks are running
task <thread 4> done
<thread 5> is finished. 1 tasks are running
<thread 4> is finished. 0 tasks are running
1 tasks are running
2 tasks are running
3 tasks are running
task <thread 8> done
task <thread 9> done
<thread 8> is finished. 2 tasks are running
task <thread 6> done
<thread 9> is finished. 1 tasks are running
2 tasks are running
<thread 6> is finished. 1 tasks are running
task <thread 7> done
<thread 7> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_small_stack (test.test_threading.ThreadTests.test_various_ops_small_stack) ... with 256 KiB thread stack size...
task <thread 0> will run for 64.8 usec
1 tasks are running
task <thread 1> will run for 25.5 usec
task <thread 0> done
task <thread 2> will run for 69.5 usec
2 tasks are running
task <thread 3> will run for 95.5 usec
task <thread 1> done
3 tasks are running
task <thread 4> will run for 26.1 usec
task <thread 5> will run for 23.0 usec
task <thread 2> done
task <thread 6> will run for 4.6 usec
task <thread 7> will run for 40.8 usec
waiting for all tasks to complete
task <thread 8> will run for 63.4 usec
<thread 0> is finished. 2 tasks are running
task <thread 9> will run for 16.8 usec
<thread 1> is finished. 1 tasks are running
<thread 2> is finished. 0 tasks are running
1 tasks are running
2 tasks are running
3 tasks are running
task <thread 4> done
<thread 4> is finished. 2 tasks are running
task <thread 3> done
task <thread 5> done
3 tasks are running
<thread 3> is finished. 2 tasks are running
<thread 5> is finished. 1 tasks are running
task <thread 6> done
2 tasks are running
3 tasks are running
<thread 6> is finished. 2 tasks are running
task <thread 7> done
task <thread 8> done
<thread 7> is finished. 1 tasks are running
2 tasks are running
<thread 8> is finished. 1 tasks are running
task <thread 9> done
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_bare_raise_in_brand_new_thread (test.test_threading.ThreadingExceptionTests.test_bare_raise_in_brand_new_thread) ... ok
test_daemonize_active_thread (test.test_threading.ThreadingExceptionTests.test_daemonize_active_thread) ... ok
test_joining_current_thread (test.test_threading.ThreadingExceptionTests.test_joining_current_thread) ... ok
test_joining_inactive_thread (test.test_threading.ThreadingExceptionTests.test_joining_inactive_thread) ... ok
test_multithread_modify_file_noerror (test.test_threading.ThreadingExceptionTests.test_multithread_modify_file_noerror) ... ok
test_print_exception (test.test_threading.ThreadingExceptionTests.test_print_exception) ... ok
test_print_exception_gh_102056 (test.test_threading.ThreadingExceptionTests.test_print_exception_gh_102056) ... ok
test_print_exception_stderr_is_none_1 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_1) ... ok
test_print_exception_stderr_is_none_2 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_2) ... ok
test_recursion_limit (test.test_threading.ThreadingExceptionTests.test_recursion_limit) ... ok
test_releasing_unacquired_lock (test.test_threading.ThreadingExceptionTests.test_releasing_unacquired_lock) ... ok
test_start_thread_again (test.test_threading.ThreadingExceptionTests.test_start_thread_again) ... ok
test_init_immutable_default_args (test.test_threading.TimerTests.test_init_immutable_default_args) ... ok

----------------------------------------------------------------------
Ran 213 tests in 35.318s

OK (skipped=4)
0:00:36 load avg: 0.54 [1/1] test_threading passed in 35.9 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 36.0 sec
Total tests: run=213 skipped=4
Total test files: run=1/1
Result: SUCCESS

I then edited the ident function to always set the upper bit to force the broken behavior I was seeing in #130115 and re-ran the tests

--- a/Python/thread_pthread.h
+++ b/Python/thread_pthread.h
@@ -315,6 +315,7 @@ _pthread_t_to_ident(pthread_t value) {
 #else
     PyThread_ident_t ident;
 #if defined(__linux__) && !defined(__GLIBC__)
+    value = (pthread_t) ((uintptr_t) value | 0x80000000);
     ident = (PyThread_ident_t) (uintptr_t) value;
     assert(pthread_equal(value, (pthread_t) (uintptr_t) ident));
 #else

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# ./python -c "import threading; print(hex(threading.current_thread().ident))"
0xc0884f34
5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# ./python -c "import threading; print(threading.current_thread() is threading.main_thread())"
True

5c3ddfcc4230:/tmp/tmp.pdpMFD/cpython# ./python -m test test_threading
Using random seed: 920746765
0:00:00 load avg: 0.42 Run 1 test sequentially in a single process
0:00:00 load avg: 0.42 [1/1] test_threading
0:00:35 load avg: 0.71 [1/1] test_threading passed in 35.3 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 35.3 sec
Total tests: run=213 skipped=4
Total test files: run=1/1
Result: SUCCESS

vfazio commented Feb 23, 2025

Copy link
Copy Markdown
Contributor Author

I still want to test on physical hardware and not on an emulated stack, but I have more confidence in this solution.

vfazio marked this pull request as ready for review February 23, 2025 02:31

vfazio commented Feb 23, 2025

Copy link
Copy Markdown
Contributor Author

forgot to revert the initial commit. Test still pass on a --with-assertions build:

05788ca7897b:/tmp/tmp.PHjnCN/cpython# ./python -m test -v test_threading
== CPython 3.14.0a5+ (heads/vfazio-thread_get_ident-dirty:dc7c4974e0, Feb 23 2025, 18:05:52) [GCC 14.2.0]
== Linux-6.8.0-51-generic-armv7l-with little-endian
== Python build: release with_assert
== cwd: /tmp/tmp.PHjnCN/cpython/build/test_python_worker_51235æ
== CPU count: 32
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3053357805
</snip>
----------------------------------------------------------------------
Ran 213 tests in 35.285s

OK (skipped=2)
0:00:35 load avg: 0.52 [1/1] test_threading passed in 35.5 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 35.7 sec
Total tests: run=213 skipped=2
Total test files: run=1/1
Result: SUCCESS

vfazio commented Feb 24, 2025

Copy link
Copy Markdown
Contributor Author

tested on hardware:

rpi-fc79b4:/var/tmp/cpython# uname -a
Linux rpi-fc79b4 6.12.13-0-rpi #1-Alpine SMP Thu Feb 13 22:14:23 UTC 2025 armv7l Linux

rpi-fc79b4:/var/tmp/cpython# cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.21.3
PRETTY_NAME="Alpine Linux v3.21"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

rpi-fc79b4:/var/tmp/cpython# ./python -m test -v test_threading
== CPython 3.14.0a5+ (heads/vfazio-thread_get_ident:dc7c4974e0, Feb 24 2025, 11:45:03) [GCC 14.2.0]
== Linux-6.12.13-0-rpi-armv7l-with little-endian
== Python build: release with_assert
== cwd: /var/tmp/cpython/build/test_python_worker_12205æ
== CPU count: 4
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 2651729950
0:00:00 load avg: 0.81 Run 1 test sequentially in a single process
0:00:00 load avg: 0.81 [1/1] test_threading
test_atexit_after_shutdown (test.test_threading.AtexitTests.test_atexit_after_shutdown) ... ok
test_atexit_called_once (test.test_threading.AtexitTests.test_atexit_called_once) ... ok
test_atexit_output (test.test_threading.AtexitTests.test_atexit_output) ... ok
test_abort (test.test_threading.BarrierTests.test_abort)
Test that an abort will put the barrier in a broken state ... ok
test_abort_and_reset (test.test_threading.BarrierTests.test_abort_and_reset)
Test that a barrier can be reset after being broken. ... ok
test_action (test.test_threading.BarrierTests.test_action)
Test the 'action' callback ... ok
test_barrier (test.test_threading.BarrierTests.test_barrier)
Test that a barrier is passed in lockstep ... ok
test_barrier_10 (test.test_threading.BarrierTests.test_barrier_10)
Test that a barrier works for 10 consecutive runs ... ok
test_constructor (test.test_threading.BarrierTests.test_constructor) ... ok
test_default_timeout (test.test_threading.BarrierTests.test_default_timeout)
Test the barrier's default timeout ... ok
test_repr (test.test_threading.BarrierTests.test_repr) ... ok
test_reset (test.test_threading.BarrierTests.test_reset)
Test that a 'reset' on a barrier frees the waiting threads ... ok
test_single_thread (test.test_threading.BarrierTests.test_single_thread) ... ok
test_timeout (test.test_threading.BarrierTests.test_timeout)
Test wait(timeout) ... ok
test_wait_return (test.test_threading.BarrierTests.test_wait_return)
test the return value from barrier.wait ... ok
test_acquire (test.test_threading.BoundedSemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.BoundedSemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.BoundedSemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.BoundedSemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.BoundedSemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.BoundedSemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.BoundedSemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.BoundedSemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.BoundedSemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.BoundedSemaphoreTests.test_with) ... ok
test__is_owned (test.test_threading.CRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.CRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.CRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.CRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.CRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.CRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.CRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.CRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.CRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.CRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.CRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.CRLockTests.test_repr) ... ok
test_signature (test.test_threading.CRLockTests.test_signature) ... ok
test_thread_leak (test.test_threading.CRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.CRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.CRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.CRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.CRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.CRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.CRLockTests.test_with) ... ok
test__is_owned (test.test_threading.ConditionAsRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.ConditionAsRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.ConditionAsRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.ConditionAsRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.ConditionAsRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.ConditionAsRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.ConditionAsRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.ConditionAsRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.ConditionAsRLockTests.test_recursion_count) ... skipped 'Condition does not expose _recursion_count()'
test_release_save_unacquired (test.test_threading.ConditionAsRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.ConditionAsRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.ConditionAsRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.ConditionAsRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.ConditionAsRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.ConditionAsRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.ConditionAsRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.ConditionAsRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.ConditionAsRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.ConditionAsRLockTests.test_with) ... ok
test_acquire (test.test_threading.ConditionTests.test_acquire) ... ok
test_notify (test.test_threading.ConditionTests.test_notify) ... ok
test_timeout (test.test_threading.ConditionTests.test_timeout) ... ok
test_unacquired_notify (test.test_threading.ConditionTests.test_unacquired_notify) ... ok
test_unacquired_wait (test.test_threading.ConditionTests.test_unacquired_wait) ... ok
test_waitfor (test.test_threading.ConditionTests.test_waitfor) ... ok
test_waitfor_timeout (test.test_threading.ConditionTests.test_waitfor_timeout) ... ok
test_at_fork_reinit (test.test_threading.EventTests.test_at_fork_reinit) ... ok
test_is_set (test.test_threading.EventTests.test_is_set) ... ok
test_notify (test.test_threading.EventTests.test_notify) ... ok
test_repr (test.test_threading.EventTests.test_repr) ... ok
test_set_and_clear (test.test_threading.EventTests.test_set_and_clear) ... ok
test_timeout (test.test_threading.EventTests.test_timeout) ... ok
test_custom_excepthook (test.test_threading.ExceptHookTests.test_custom_excepthook) ... ok
test_custom_excepthook_fail (test.test_threading.ExceptHookTests.test_custom_excepthook_fail) ... ok
test_excepthook (test.test_threading.ExceptHookTests.test_excepthook) ... ok
test_excepthook_thread_None (test.test_threading.ExceptHookTests.test_excepthook_thread_None) ... ok
test_original_excepthook (test.test_threading.ExceptHookTests.test_original_excepthook) ... ok
test_system_exit (test.test_threading.ExceptHookTests.test_system_exit) ... ok
test_can_interrupt_tight_loops (test.test_threading.InterruptMainTests.test_can_interrupt_tight_loops) ... ok
test_interrupt_main_invalid_signal (test.test_threading.InterruptMainTests.test_interrupt_main_invalid_signal) ... ok
test_interrupt_main_mainthread (test.test_threading.InterruptMainTests.test_interrupt_main_mainthread) ... ok
test_interrupt_main_noerror (test.test_threading.InterruptMainTests.test_interrupt_main_noerror) ... ok
test_interrupt_main_subthread (test.test_threading.InterruptMainTests.test_interrupt_main_subthread) ... ok
test_interrupt_main_with_signal_handler (test.test_threading.InterruptMainTests.test_interrupt_main_with_signal_handler) ... ok
test_acquire_contended (test.test_threading.LockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.LockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.LockTests.test_acquire_release) ... ok
test_at_fork_reinit (test.test_threading.LockTests.test_at_fork_reinit) ... ok
test_constructor (test.test_threading.LockTests.test_constructor) ... ok
test_different_thread (test.test_threading.LockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.LockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.LockTests.test_reacquire) ... ok
test_repr (test.test_threading.LockTests.test_repr) ... ok
test_state_after_timeout (test.test_threading.LockTests.test_state_after_timeout) ... ok
test_thread_leak (test.test_threading.LockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.LockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.LockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.LockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.LockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.LockTests.test_weakref_exists) ... ok
test_with (test.test_threading.LockTests.test_with) ... ok
test__all__ (test.test_threading.MiscTestCase.test__all__) ... ok
test_change_name (test.test_threading.MiscTestCase.test_change_name) ... ok
test_set_name (test.test_threading.MiscTestCase.test_set_name) ... ok
test__is_owned (test.test_threading.PyRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.PyRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.PyRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.PyRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.PyRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.PyRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.PyRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.PyRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.PyRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.PyRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.PyRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.PyRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.PyRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.PyRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.PyRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.PyRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.PyRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.PyRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.PyRLockTests.test_with) ... ok
test_acquire (test.test_threading.SemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.SemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.SemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.SemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.SemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.SemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.SemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.SemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.SemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.SemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.SemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.SemaphoreTests.test_with) ... ok
test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadingTests.test_daemon_threads_fatal_error) ... ok
test_daemon_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_daemon_threads_not_allowed) ... ok
test_threads_join (test.test_threading.SubinterpThreadingTests.test_threads_join) ... ok
test_threads_join_2 (test.test_threading.SubinterpThreadingTests.test_threads_join_2) ... ok
test_threads_join_with_no_main (test.test_threading.SubinterpThreadingTests.test_threads_join_with_no_main) ... ok
test_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_threads_not_allowed) ... ok
test_1_join_on_shutdown (test.test_threading.ThreadJoinOnShutdown.test_1_join_on_shutdown) ... ok
test_2_join_in_forked_process (test.test_threading.ThreadJoinOnShutdown.test_2_join_in_forked_process) ... ok
test_3_join_in_forked_from_thread (test.test_threading.ThreadJoinOnShutdown.test_3_join_in_forked_from_thread) ... ok
test_4_daemon_threads (test.test_threading.ThreadJoinOnShutdown.test_4_daemon_threads) ... ok
test_clear_threads_states_after_fork (test.test_threading.ThreadJoinOnShutdown.test_clear_threads_states_after_fork) ...
 ok
test_reinit_tls_after_fork (test.test_threading.ThreadJoinOnShutdown.test_reinit_tls_after_fork) ... ok
test_thread_from_thread (test.test_threading.ThreadJoinOnShutdown.test_thread_from_thread) ... ok
test_BoundedSemaphore_limit (test.test_threading.ThreadTests.test_BoundedSemaphore_limit) ... ok
test_PyThreadState_SetAsyncExc (test.test_threading.ThreadTests.test_PyThreadState_SetAsyncExc) ...     started worker thread
    trying nonsensical thread id
    waiting for worker thread to get started
    verifying worker hasn't exited
    attempting to raise asynch exception in worker
    waiting for worker to say it caught the exception
    all OK -- joining worker
ok
test_args_argument (test.test_threading.ThreadTests.test_args_argument) ... ok
test_boolean_target (test.test_threading.ThreadTests.test_boolean_target) ... ok
test_daemon_param (test.test_threading.ThreadTests.test_daemon_param) ... ok
test_dummy_thread_after_fork (test.test_threading.ThreadTests.test_dummy_thread_after_fork) ... ok
test_enumerate_after_join (test.test_threading.ThreadTests.test_enumerate_after_join) ... ok
test_finalization_shutdown (test.test_threading.ThreadTests.test_finalization_shutdown) ... ok
test_finalize_daemon_thread_hang (test.test_threading.ThreadTests.test_finalize_daemon_thread_hang) ... ok
test_finalize_running_thread (test.test_threading.ThreadTests.test_finalize_running_thread) ... ok
test_finalize_with_trace (test.test_threading.ThreadTests.test_finalize_with_trace) ... ok
test_foreign_thread (test.test_threading.ThreadTests.test_foreign_thread) ... ok
test_frame_tstate_tracing (test.test_threading.ThreadTests.test_frame_tstate_tracing) ... ok
test_getprofile (test.test_threading.ThreadTests.test_getprofile) ... ok
test_getprofile_all_threads (test.test_threading.ThreadTests.test_getprofile_all_threads) ... ok
test_gettrace (test.test_threading.ThreadTests.test_gettrace) ... ok
test_gettrace_all_threads (test.test_threading.ThreadTests.test_gettrace_all_threads) ... ok
test_ident_of_no_threading_threads (test.test_threading.ThreadTests.test_ident_of_no_threading_threads) ... ok
test_import_from_another_thread (test.test_threading.ThreadTests.test_import_from_another_thread) ... ok
test_is_alive_after_fork (test.test_threading.ThreadTests.test_is_alive_after_fork) ... ok
test_join_from_multiple_threads (test.test_threading.ThreadTests.test_join_from_multiple_threads) ... ok
test_join_nondaemon_on_shutdown (test.test_threading.ThreadTests.test_join_nondaemon_on_shutdown) ... ok
test_join_with_timeout (test.test_threading.ThreadTests.test_join_with_timeout) ... ok
test_leak_without_join (test.test_threading.ThreadTests.test_leak_without_join) ... ok
test_limbo_cleanup (test.test_threading.ThreadTests.test_limbo_cleanup) ... ok
test_locals_at_exit (test.test_threading.ThreadTests.test_locals_at_exit) ... ok
test_lock_no_args (test.test_threading.ThreadTests.test_lock_no_args) ... ok
test_lock_no_subclass (test.test_threading.ThreadTests.test_lock_no_subclass) ... ok
test_lock_or_none (test.test_threading.ThreadTests.test_lock_or_none) ... ok
test_main_thread (test.test_threading.ThreadTests.test_main_thread) ... ok
test_main_thread_after_fork (test.test_threading.ThreadTests.test_main_thread_after_fork) ... ok
test_main_thread_after_fork_from_dummy_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_dummy_thread) ... ok
test_main_thread_after_fork_from_foreign_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_foreign_thread) ... ok
test_main_thread_after_fork_from_nonmain_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_nonmain_thread) ... ok
test_main_thread_during_shutdown (test.test_threading.ThreadTests.test_main_thread_during_shutdown) ... ok
test_name (test.test_threading.ThreadTests.test_name) ... ok
test_no_refcycle_through_target (test.test_threading.ThreadTests.test_no_refcycle_through_target) ... ok
test_old_threading_api (test.test_threading.ThreadTests.test_old_threading_api) ... ok
test_repr_daemon (test.test_threading.ThreadTests.test_repr_daemon) ... ok
test_repr_stopped (test.test_threading.ThreadTests.test_repr_stopped) ... ok
test_start_new_thread_at_finalization (test.test_threading.ThreadTests.test_start_new_thread_at_finalization) ... ok
test_start_new_thread_failed (test.test_threading.ThreadTests.test_start_new_thread_failed) ... skipped 'RLIMIT_NPROC had no effect; probably superuser'
test_various_ops (test.test_threading.ThreadTests.test_various_ops) ... task <thread 0> will run for 88.0 usec
task <thread 1> will run for 34.8 usec
1 tasks are running
task <thread 2> will run for 52.4 usec
task <thread 4> will run for 58.5 usec
2 tasks are running
task <thread 5> will run for 22.7 usec
task <thread 6> will run for 54.6 usec
task <thread 3> will run for 55.7 usec
task <thread 7> will run for 35.8 usec
3 tasks are running
task <thread 8> will run for 71.6 usec
task <thread 9> will run for 91.9 usec
task <thread 1> done
task <thread 2> done
task <thread 0> done
waiting for all tasks to complete
<thread 1> is finished. 2 tasks are running
<thread 2> is finished. 1 tasks are running
<thread 0> is finished. 0 tasks are running
1 tasks are running
2 tasks are running
task <thread 4> done
task <thread 5> done
3 tasks are running
<thread 4> is finished. 2 tasks are running
<thread 5> is finished. 1 tasks are running
task <thread 6> done
2 tasks are running
3 tasks are running
task <thread 3> done
<thread 6> is finished. 2 tasks are running
task <thread 7> done
<thread 3> is finished. 1 tasks are running
<thread 7> is finished. 0 tasks are running
1 tasks are running
2 tasks are running
task <thread 8> done
task <thread 9> done
<thread 8> is finished. 1 tasks are running
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_large_stack (test.test_threading.ThreadTests.test_various_ops_large_stack) ... with 1 MiB thread stack size...
task <thread 0> will run for 54.4 usec
1 tasks are running
task <thread 1> will run for 8.1 usec
task <thread 0> done
2 tasks are running
task <thread 3> will run for 58.1 usec
task <thread 4> will run for 10.1 usec
<thread 0> is finished. 1 tasks are running
task <thread 5> will run for 44.9 usec
task <thread 6> will run for 7.0 usec
task <thread 7> will run for 66.0 usec
2 tasks are running
3 tasks are running
task <thread 9> will run for 17.6 usec
task <thread 2> will run for 81.3 usec
task <thread 8> will run for 78.9 usec
waiting for all tasks to complete
task <thread 1> done
task <thread 3> done
<thread 1> is finished. 2 tasks are running
task <thread 4> done
<thread 3> is finished. 1 tasks are running
2 tasks are running
<thread 4> is finished. 1 tasks are running
task <thread 5> done
2 tasks are running
3 tasks are running
<thread 5> is finished. 2 tasks are running
task <thread 6> done
task <thread 7> done
3 tasks are running
<thread 6> is finished. 2 tasks are running
<thread 7> is finished. 1 tasks are running
task <thread 9> done
2 tasks are running
<thread 9> is finished. 1 tasks are running
task <thread 8> done
2 tasks are running
task <thread 2> done
<thread 8> is finished. 1 tasks are running
<thread 2> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_small_stack (test.test_threading.ThreadTests.test_various_ops_small_stack) ... with 256 KiB thread stack size...
task <thread 0> will run for 58.9 usec
1 tasks are running
task <thread 1> will run for 55.7 usec
task <thread 0> done
2 tasks are running
task <thread 2> will run for 20.3 usec
task <thread 1> done
task <thread 3> will run for 78.7 usec
<thread 0> is finished. 1 tasks are running
task <thread 5> will run for 86.6 usec
task <thread 6> will run for 65.3 usec
2 tasks are running
task <thread 4> will run for 18.3 usec
task <thread 7> will run for 88.4 usec
<thread 1> is finished. 1 tasks are running
task <thread 2> done
task <thread 9> will run for 19.4 usec
waiting for all tasks to complete
task <thread 8> will run for 13.2 usec
2 tasks are running
3 tasks are running
task <thread 3> done
<thread 2> is finished. 2 tasks are running
task <thread 5> done
3 tasks are running
<thread 3> is finished. 2 tasks are running
task <thread 6> done
<thread 5> is finished. 1 tasks are running
2 tasks are running
<thread 6> is finished. 1 tasks are running
task <thread 7> done
2 tasks are running
task <thread 4> done
3 tasks are running
<thread 7> is finished. 2 tasks are running
task <thread 9> done
<thread 4> is finished. 1 tasks are running
2 tasks are running
task <thread 8> done
<thread 9> is finished. 1 tasks are running
<thread 8> is finished. 0 tasks are running
all tasks done
ok
test_bare_raise_in_brand_new_thread (test.test_threading.ThreadingExceptionTests.test_bare_raise_in_brand_new_thread) ... ok
test_daemonize_active_thread (test.test_threading.ThreadingExceptionTests.test_daemonize_active_thread) ... ok
test_joining_current_thread (test.test_threading.ThreadingExceptionTests.test_joining_current_thread) ... ok
test_joining_inactive_thread (test.test_threading.ThreadingExceptionTests.test_joining_inactive_thread) ... ok
test_multithread_modify_file_noerror (test.test_threading.ThreadingExceptionTests.test_multithread_modify_file_noerror) ... ok
test_print_exception (test.test_threading.ThreadingExceptionTests.test_print_exception) ... ok
test_print_exception_gh_102056 (test.test_threading.ThreadingExceptionTests.test_print_exception_gh_102056) ... ok
test_print_exception_stderr_is_none_1 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_1) ... ok
test_print_exception_stderr_is_none_2 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_2) ... ok
test_recursion_limit (test.test_threading.ThreadingExceptionTests.test_recursion_limit) ... ok
test_releasing_unacquired_lock (test.test_threading.ThreadingExceptionTests.test_releasing_unacquired_lock) ... ok
test_start_thread_again (test.test_threading.ThreadingExceptionTests.test_start_thread_again) ... ok
test_init_immutable_default_args (test.test_threading.TimerTests.test_init_immutable_default_args) ... ok

----------------------------------------------------------------------
Ran 213 tests in 52.328s

OK (skipped=2)
0:00:53 load avg: 0.93 [1/1] test_threading passed in 53.1 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 53.5 sec
Total tests: run=213 skipped=2
Total test files: run=1/1
Result: SUCCESS

vfazio commented Feb 24, 2025

Copy link
Copy Markdown
Contributor Author

repeated with high bit set:

rpi-fc79b4:/var/tmp/cpython# ./python -c "import threading; print(hex(threading.current_thread().ident))"
0xf6f0cf34
rpi-fc79b4:/var/tmp/cpython# ./python -c "import threading; print(threading.current_thread() is threading.main_thread())"
True

rpi-fc79b4:/var/tmp/cpython# ./python -m test test_threading
Using random seed: 79905904
0:00:00 load avg: 0.46 Run 1 test sequentially in a single process
0:00:00 load avg: 0.46 [1/1] test_threading
0:00:51 load avg: 0.87 [1/1] test_threading passed in 51.1 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 51.1 sec
Total tests: run=213 skipped=2
Total test files: run=1/1
Result: SUCCESS

vfazio commented Mar 31, 2025

Copy link
Copy Markdown
Contributor Author

@mpage It's been a while since we discussed #130115 so I'm interested in your thoughts here.

This PR addresses the primary issue, can be backported, and, as far as I can tell, doesn't introduce any regressions.

I took a peek at some other changes last night and I'm curious if you think it would be worthwhile to reuse a similar data structure as the min heap for thread identifiers long term. It would recycle them faster, but so long as callers aren't assuming a specific lifetime maybe that's OK. I guess other assumptions are that thread IDs are tied to a specific interpreter instance and can't be passed around between them since they're no longer system level identifiers

mpage requested review from Yhg1s and pitrou March 31, 2025 21:16
mpage reviewed Mar 31, 2025
View reviewed changes

mpage left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This looks mostly ok to me, but the use of _pthread_t_to_ident in PyThread_get_thread_ident_ex() makes me a little nervous when SIZEOF_PTHREAD_T > SIZEOF_LONG (see inline comment). I would probably take the approach of adding a second field to PyRuntime (even though it's gross) or potentially reverting to the previous behavior of using set_ident for the main thread. I'd like to see what others think. I've added a couple of folks who have worked in this area in the past as reviewers.

Comment thread Python/thread_pthread.h
threadid = pthread_self();
assert(threadid == (pthread_t) (PyThread_ident_t) threadid);
return (PyThread_ident_t) threadid;
return _pthread_t_to_ident(threadid);

mpage Mar 31, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think this is potentially lossy compared to the previous version if SIZEOF_PTHREAD_T > SIZEOF_LONG. Previously we would cast directly to a PyThread_ident_t (an unsigned long long), whereas we now cast through an unsigned long.

vfazio Mar 31, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think I understand your concern.

This was an edge case in PyThread_start_new_thread, my guess is that it was there for the situations where pthread_t was not a ulong and was, instead, a pointer or struct.

#if SIZEOF_PTHREAD_T <= SIZEOF_LONG
    return (unsigned long) th;
#else
    return (unsigned long) *(unsigned long *) &th;

It's probably fine to drop the condition in _pthread_t_to_ident and let the caller here truncate it since this API is specifically returning a ULONG and not a PyThread_ident_t.

Looking at the history:

2565bff

635f6fb

The cast through ulong* was for Alpha OSF which was dropped from PEP 11 a few years ago (CPython 3.3) https://bugs.python.org/issue8606. My guess is it was also defined as a pointer or struct type and this was a way to work around it by returning at least long bytes.

I think if we find other platforms where we need to support this workaround, they can be chained to the MUSL #ifdef I think. The only time it would maybe be a problem is if sizeof(uintptr_t) < sizeof(ulong).

If others agree, I can drop the condition so the function looks like so:

static PyThread_ident_t
_pthread_t_to_ident(pthread_t value) {
    PyThread_ident_t ident;
#if defined(__linux__) && !defined(__GLIBC__)
    ident = (PyThread_ident_t) (uintptr_t) value;
    assert(pthread_equal(value, (pthread_t) (uintptr_t) ident));
#else
    ident = (PyThread_ident_t) value;
    assert(pthread_equal(value, (pthread_t) ident));
#endif
    return ident;
}

I do not suggest this as a long term solution; I do think we need to work towards making this opaque. I'm just trying to find something that is a stop-gap that can be ported back with relative ease that doesn't cause a regression.

mpage commented Mar 31, 2025

Copy link
Copy Markdown
Contributor

I took a peek at some other changes last night and I'm curious if you think it would be worthwhile to reuse a similar data structure as the min heap for thread identifiers long term.

Yes, something like that could make sense.

It would recycle them faster, but so long as callers aren't assuming a specific lifetime maybe that's OK.

I think that is already true now (you can't make assumptions about the lifetime of pthread identifiers).

I guess other assumptions are that thread IDs are tied to a specific interpreter instance and can't be passed around between them since they're no longer system level identifiers

Yeah, depending on how we chose to manage thread IDs that could be true. We could choose to manage the the pool of thread IDs per runtime, which would preserve the existing namespacing.

pitrou reviewed Apr 1, 2025
View reviewed changes
Comment thread Python/thread_pthread.h Outdated
static PyThread_ident_t
_pthread_t_to_ident(pthread_t value) {
#if SIZEOF_PTHREAD_T > SIZEOF_LONG
return (PyThread_ident_t) *(unsigned long *) &value;

pitrou Apr 1, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why go through a pointer? This is cryptic and also uncomfortably fragile (what about big-endian platforms?).

I would suggest perhaps something like:

static PyThread_ident_t
_pthread_t_to_ident(pthread_t value) {
    // Avoid sign-extension when converting to a larger int type
#if SIZEOF_PTHREAD_T == SIZEOF_VOID_P
    return (uintptr_t) value;
#elif SIZEOF_PTHREAD_T == SIZEOF_LONG
    return (unsigned long) value;
#elif SIZEOF_PTHREAD_T == SIZEOF_INT
    return (unsigned int) value;
#elif SIZEOF_PTHREAD_T == SIZEOF_LONG_LONG
    return (unsigned long long) value;
#else
#error "Unsupported SIZEOF_PTHREAD_T value"
#endif
}

vfazio Apr 1, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

That was legacy code that I can certainly drop (as mentioned above) . This is definitely simpler, I'll give it a spin and push it if tests pass.

vfazio Apr 2, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I've updated the branch to this implementation. The unit tests continue to pass AFAICT.

vfazio commented Apr 2, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Testing on musl + armv7 on RPi passes with the implementation suggested with and without the high bit set

localhost:~/development/cpython# ./python -m test -v test_threading
== CPython 3.14.0a6+ (heads/vfazio-thread_get_ident:4ae6d7e83d6, Apr 3 2025, 07:02:00) [GCC 14.2.0]
== Linux-6.12.13-0-rpi-armv7l-with-musl1.2.5 little-endian
== Python build: release
== cwd: /root/development/cpython/build/test_python_worker_27815æ
== CPU count: 4
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 2337934113
0:00:00 load avg: 0.55 Run 1 test sequentially in a single process
0:00:00 load avg: 0.55 [1/1] test_threading
test_atexit_after_shutdown (test.test_threading.AtexitTests.test_atexit_after_shutdown) ... ok
test_atexit_called_once (test.test_threading.AtexitTests.test_atexit_called_once) ... ok
test_atexit_output (test.test_threading.AtexitTests.test_atexit_output) ... ok
test_abort (test.test_threading.BarrierTests.test_abort)
Test that an abort will put the barrier in a broken state ... ok
test_abort_and_reset (test.test_threading.BarrierTests.test_abort_and_reset)
Test that a barrier can be reset after being broken. ... ok
test_action (test.test_threading.BarrierTests.test_action)
Test the 'action' callback ... ok
test_barrier (test.test_threading.BarrierTests.test_barrier)
Test that a barrier is passed in lockstep ... ok
test_barrier_10 (test.test_threading.BarrierTests.test_barrier_10)
Test that a barrier works for 10 consecutive runs ... ok
test_constructor (test.test_threading.BarrierTests.test_constructor) ... ok
test_default_timeout (test.test_threading.BarrierTests.test_default_timeout)
Test the barrier's default timeout ... ok
test_repr (test.test_threading.BarrierTests.test_repr) ... ok
test_reset (test.test_threading.BarrierTests.test_reset)
Test that a 'reset' on a barrier frees the waiting threads ... ok
test_single_thread (test.test_threading.BarrierTests.test_single_thread) ... ok
test_timeout (test.test_threading.BarrierTests.test_timeout)
Test wait(timeout) ... ok
test_wait_return (test.test_threading.BarrierTests.test_wait_return)
test the return value from barrier.wait ... ok
test_acquire (test.test_threading.BoundedSemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.BoundedSemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.BoundedSemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.BoundedSemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.BoundedSemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.BoundedSemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.BoundedSemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.BoundedSemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.BoundedSemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.BoundedSemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.BoundedSemaphoreTests.test_with) ... ok
test__is_owned (test.test_threading.CRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.CRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.CRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.CRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.CRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.CRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.CRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.CRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.CRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.CRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.CRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.CRLockTests.test_repr) ... ok
test_signature (test.test_threading.CRLockTests.test_signature) ... ok
test_thread_leak (test.test_threading.CRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.CRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.CRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.CRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.CRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.CRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.CRLockTests.test_with) ... ok
test__is_owned (test.test_threading.ConditionAsRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.ConditionAsRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.ConditionAsRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.ConditionAsRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.ConditionAsRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.ConditionAsRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.ConditionAsRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.ConditionAsRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.ConditionAsRLockTests.test_recursion_count) ... skipped 'Condition does not expose _recursion_count()'
test_release_save_unacquired (test.test_threading.ConditionAsRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.ConditionAsRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.ConditionAsRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.ConditionAsRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.ConditionAsRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.ConditionAsRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.ConditionAsRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.ConditionAsRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.ConditionAsRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.ConditionAsRLockTests.test_with) ... ok
test_acquire (test.test_threading.ConditionTests.test_acquire) ... ok
test_notify (test.test_threading.ConditionTests.test_notify) ... ok
test_timeout (test.test_threading.ConditionTests.test_timeout) ... ok
test_unacquired_notify (test.test_threading.ConditionTests.test_unacquired_notify) ... ok
test_unacquired_wait (test.test_threading.ConditionTests.test_unacquired_wait) ... ok
test_waitfor (test.test_threading.ConditionTests.test_waitfor) ... ok
test_waitfor_timeout (test.test_threading.ConditionTests.test_waitfor_timeout) ... ok
test_at_fork_reinit (test.test_threading.EventTests.test_at_fork_reinit) ... ok
test_is_set (test.test_threading.EventTests.test_is_set) ... ok
test_notify (test.test_threading.EventTests.test_notify) ... ok
test_repr (test.test_threading.EventTests.test_repr) ... ok
test_set_and_clear (test.test_threading.EventTests.test_set_and_clear) ... ok
test_timeout (test.test_threading.EventTests.test_timeout) ... ok
test_custom_excepthook (test.test_threading.ExceptHookTests.test_custom_excepthook) ... ok
test_custom_excepthook_fail (test.test_threading.ExceptHookTests.test_custom_excepthook_fail) ... ok
test_excepthook (test.test_threading.ExceptHookTests.test_excepthook) ... ok
test_excepthook_thread_None (test.test_threading.ExceptHookTests.test_excepthook_thread_None) ... ok
test_original_excepthook (test.test_threading.ExceptHookTests.test_original_excepthook) ... ok
test_system_exit (test.test_threading.ExceptHookTests.test_system_exit) ... ok
test_can_interrupt_tight_loops (test.test_threading.InterruptMainTests.test_can_interrupt_tight_loops) ... ok
test_interrupt_main_invalid_signal (test.test_threading.InterruptMainTests.test_interrupt_main_invalid_signal) ... ok
test_interrupt_main_mainthread (test.test_threading.InterruptMainTests.test_interrupt_main_mainthread) ... ok
test_interrupt_main_noerror (test.test_threading.InterruptMainTests.test_interrupt_main_noerror) ... ok
test_interrupt_main_subthread (test.test_threading.InterruptMainTests.test_interrupt_main_subthread) ... ok
test_interrupt_main_with_signal_handler (test.test_threading.InterruptMainTests.test_interrupt_main_with_signal_handler) ... ok
test_acquire_contended (test.test_threading.LockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.LockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.LockTests.test_acquire_release) ... ok
test_at_fork_reinit (test.test_threading.LockTests.test_at_fork_reinit) ... ok
test_constructor (test.test_threading.LockTests.test_constructor) ... ok
test_different_thread (test.test_threading.LockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.LockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.LockTests.test_reacquire) ... ok
test_repr (test.test_threading.LockTests.test_repr) ... ok
test_state_after_timeout (test.test_threading.LockTests.test_state_after_timeout) ... ok
test_thread_leak (test.test_threading.LockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.LockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.LockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.LockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.LockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.LockTests.test_weakref_exists) ... ok
test_with (test.test_threading.LockTests.test_with) ... ok
test__all__ (test.test_threading.MiscTestCase.test__all__) ... ok
test_change_name (test.test_threading.MiscTestCase.test_change_name) ... ok
test_set_name (test.test_threading.MiscTestCase.test_set_name) ... ok
test__is_owned (test.test_threading.PyRLockTests.test__is_owned) ... ok
test_acquire_contended (test.test_threading.PyRLockTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.PyRLockTests.test_acquire_destroy) ... ok
test_acquire_release (test.test_threading.PyRLockTests.test_acquire_release) ... ok
test_constructor (test.test_threading.PyRLockTests.test_constructor) ... ok
test_different_thread (test.test_threading.PyRLockTests.test_different_thread) ... ok
test_locked_repr (test.test_threading.PyRLockTests.test_locked_repr) ... ok
test_reacquire (test.test_threading.PyRLockTests.test_reacquire) ... ok
test_recursion_count (test.test_threading.PyRLockTests.test_recursion_count) ... ok
test_release_save_unacquired (test.test_threading.PyRLockTests.test_release_save_unacquired) ... ok
test_release_unacquired (test.test_threading.PyRLockTests.test_release_unacquired) ... ok
test_repr (test.test_threading.PyRLockTests.test_repr) ... ok
test_thread_leak (test.test_threading.PyRLockTests.test_thread_leak) ... ok
test_timeout (test.test_threading.PyRLockTests.test_timeout) ... ok
test_try_acquire (test.test_threading.PyRLockTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.PyRLockTests.test_try_acquire_contended) ... ok
test_weakref_deleted (test.test_threading.PyRLockTests.test_weakref_deleted) ... ok
test_weakref_exists (test.test_threading.PyRLockTests.test_weakref_exists) ... ok
test_with (test.test_threading.PyRLockTests.test_with) ... ok
test_acquire (test.test_threading.SemaphoreTests.test_acquire) ... ok
test_acquire_contended (test.test_threading.SemaphoreTests.test_acquire_contended) ... ok
test_acquire_destroy (test.test_threading.SemaphoreTests.test_acquire_destroy) ... ok
test_acquire_timeout (test.test_threading.SemaphoreTests.test_acquire_timeout) ... ok
test_constructor (test.test_threading.SemaphoreTests.test_constructor) ... ok
test_default_value (test.test_threading.SemaphoreTests.test_default_value) ... ok
test_multirelease (test.test_threading.SemaphoreTests.test_multirelease) ... ok
test_release_unacquired (test.test_threading.SemaphoreTests.test_release_unacquired) ... ok
test_repr (test.test_threading.SemaphoreTests.test_repr) ... ok
test_try_acquire (test.test_threading.SemaphoreTests.test_try_acquire) ... ok
test_try_acquire_contended (test.test_threading.SemaphoreTests.test_try_acquire_contended) ... ok
test_with (test.test_threading.SemaphoreTests.test_with) ... ok
test_daemon_threads_fatal_error (test.test_threading.SubinterpThreadingTests.test_daemon_threads_fatal_error) ... ok
test_daemon_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_daemon_threads_not_allowed) ... ok
test_threads_join (test.test_threading.SubinterpThreadingTests.test_threads_join) ... ok
test_threads_join_2 (test.test_threading.SubinterpThreadingTests.test_threads_join_2) ... ok
test_threads_join_with_no_main (test.test_threading.SubinterpThreadingTests.test_threads_join_with_no_main) ... ok
test_threads_not_allowed (test.test_threading.SubinterpThreadingTests.test_threads_not_allowed) ... ok
test_1_join_on_shutdown (test.test_threading.ThreadJoinOnShutdown.test_1_join_on_shutdown) ... ok
test_2_join_in_forked_process (test.test_threading.ThreadJoinOnShutdown.test_2_join_in_forked_process) ... ok
test_3_join_in_forked_from_thread (test.test_threading.ThreadJoinOnShutdown.test_3_join_in_forked_from_thread) ... ok
test_4_daemon_threads (test.test_threading.ThreadJoinOnShutdown.test_4_daemon_threads) ... ok
test_clear_threads_states_after_fork (test.test_threading.ThreadJoinOnShutdown.test_clear_threads_states_after_fork) ... ok
test_reinit_tls_after_fork (test.test_threading.ThreadJoinOnShutdown.test_reinit_tls_after_fork) ... ok
test_thread_from_thread (test.test_threading.ThreadJoinOnShutdown.test_thread_from_thread) ... ok
test_BoundedSemaphore_limit (test.test_threading.ThreadTests.test_BoundedSemaphore_limit) ... ok
test_PyThreadState_SetAsyncExc (test.test_threading.ThreadTests.test_PyThreadState_SetAsyncExc) ...     started worker thread
    trying nonsensical thread id
    waiting for worker thread to get started
    verifying worker hasn't exited
    attempting to raise asynch exception in worker
    waiting for worker to say it caught the exception
    all OK -- joining worker
ok
test_args_argument (test.test_threading.ThreadTests.test_args_argument) ... ok
test_boolean_target (test.test_threading.ThreadTests.test_boolean_target) ... ok
test_daemon_param (test.test_threading.ThreadTests.test_daemon_param) ... ok
test_dummy_thread_after_fork (test.test_threading.ThreadTests.test_dummy_thread_after_fork) ... ok
test_enumerate_after_join (test.test_threading.ThreadTests.test_enumerate_after_join) ... ok
test_finalization_shutdown (test.test_threading.ThreadTests.test_finalization_shutdown) ... ok
test_finalize_daemon_thread_hang (test.test_threading.ThreadTests.test_finalize_daemon_thread_hang) ... ok
test_finalize_running_thread (test.test_threading.ThreadTests.test_finalize_running_thread) ... ok
test_finalize_with_trace (test.test_threading.ThreadTests.test_finalize_with_trace) ... ok
test_foreign_thread (test.test_threading.ThreadTests.test_foreign_thread) ... ok
test_frame_tstate_tracing (test.test_threading.ThreadTests.test_frame_tstate_tracing) ... ok
test_getprofile (test.test_threading.ThreadTests.test_getprofile) ... ok
test_getprofile_all_threads (test.test_threading.ThreadTests.test_getprofile_all_threads) ... ok
test_gettrace (test.test_threading.ThreadTests.test_gettrace) ... ok
test_gettrace_all_threads (test.test_threading.ThreadTests.test_gettrace_all_threads) ... ok
test_ident_of_no_threading_threads (test.test_threading.ThreadTests.test_ident_of_no_threading_threads) ... ok
test_import_from_another_thread (test.test_threading.ThreadTests.test_import_from_another_thread) ... ok
test_is_alive_after_fork (test.test_threading.ThreadTests.test_is_alive_after_fork) ... ok
test_join_from_multiple_threads (test.test_threading.ThreadTests.test_join_from_multiple_threads) ... ok
test_join_nondaemon_on_shutdown (test.test_threading.ThreadTests.test_join_nondaemon_on_shutdown) ... ok
test_join_with_timeout (test.test_threading.ThreadTests.test_join_with_timeout) ... ok
test_leak_without_join (test.test_threading.ThreadTests.test_leak_without_join) ... ok
test_limbo_cleanup (test.test_threading.ThreadTests.test_limbo_cleanup) ... ok
test_locals_at_exit (test.test_threading.ThreadTests.test_locals_at_exit) ... ok
test_lock_no_args (test.test_threading.ThreadTests.test_lock_no_args) ... ok
test_lock_no_subclass (test.test_threading.ThreadTests.test_lock_no_subclass) ... ok
test_lock_or_none (test.test_threading.ThreadTests.test_lock_or_none) ... ok
test_main_thread (test.test_threading.ThreadTests.test_main_thread) ... ok
test_main_thread_after_fork (test.test_threading.ThreadTests.test_main_thread_after_fork) ... ok
test_main_thread_after_fork_from_dummy_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_dummy_thread) ... ok
test_main_thread_after_fork_from_foreign_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_foreign_thread) ... ok
test_main_thread_after_fork_from_nonmain_thread (test.test_threading.ThreadTests.test_main_thread_after_fork_from_nonmain_thread) ... ok
test_main_thread_during_shutdown (test.test_threading.ThreadTests.test_main_thread_during_shutdown) ... ok
test_name (test.test_threading.ThreadTests.test_name) ... ok
test_no_refcycle_through_target (test.test_threading.ThreadTests.test_no_refcycle_through_target) ... ok
test_old_threading_api (test.test_threading.ThreadTests.test_old_threading_api) ... ok
test_repr_daemon (test.test_threading.ThreadTests.test_repr_daemon) ... ok
test_repr_stopped (test.test_threading.ThreadTests.test_repr_stopped) ... ok
test_start_new_thread_at_finalization (test.test_threading.ThreadTests.test_start_new_thread_at_finalization) ... ok
test_start_new_thread_failed (test.test_threading.ThreadTests.test_start_new_thread_failed) ... skipped 'RLIMIT_NPROC had no effect; probably superuser'
test_various_ops (test.test_threading.ThreadTests.test_various_ops) ... task <thread 0> will run for 36.1 usec
1 tasks are running
task <thread 1> will run for 59.3 usec
task <thread 0> done
2 tasks are running
<thread 0> is finished. 1 tasks are running
task <thread 2> will run for 14.8 usec
task <thread 1> done
2 tasks are running
task <thread 4> will run for 80.2 usec
task <thread 5> will run for 26.3 usec
<thread 1> is finished. 1 tasks are running
task <thread 6> will run for 48.8 usec
task <thread 7> will run for 43.6 usec
task <thread 8> will run for 12.1 usec
task <thread 3> will run for 26.5 usec
task <thread 2> done
2 tasks are running
task <thread 9> will run for 96.7 usec
3 tasks are running
<thread 2> is finished. 2 tasks are running
task <thread 4> done
task <thread 5> done
3 tasks are running
waiting for all tasks to complete
<thread 4> is finished. 2 tasks are running
task <thread 6> done
<thread 5> is finished. 1 tasks are running
2 tasks are running
<thread 6> is finished. 1 tasks are running
2 tasks are running
task <thread 8> done
3 tasks are running
task <thread 3> done
<thread 3> is finished. 2 tasks are running
task <thread 7> done
3 tasks are running
<thread 8> is finished. 2 tasks are running
task <thread 9> done
<thread 7> is finished. 1 tasks are running
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_large_stack (test.test_threading.ThreadTests.test_various_ops_large_stack) ... with 1 MiB thread stack size...
task <thread 0> will run for 36.5 usec
1 tasks are running
task <thread 1> will run for 24.0 usec
task <thread 2> will run for 41.1 usec
2 tasks are running
task <thread 0> done
task <thread 3> will run for 75.6 usec
task <thread 4> will run for 3.5 usec
task <thread 1> done
task <thread 6> will run for 5.7 usec
task <thread 7> will run for 16.7 usec
task <thread 8> will run for 22.0 usec
task <thread 9> will run for 27.7 usec
waiting for all tasks to complete
task <thread 5> will run for 42.2 usec
3 tasks are running
<thread 0> is finished. 2 tasks are running
task <thread 2> done
<thread 1> is finished. 1 tasks are running
2 tasks are running
<thread 2> is finished. 1 tasks are running
task <thread 3> done
2 tasks are running
3 tasks are running
task <thread 6> done
<thread 3> is finished. 2 tasks are running
task <thread 4> done
<thread 6> is finished. 1 tasks are running
2 tasks are running
task <thread 7> done
<thread 4> is finished. 1 tasks are running
2 tasks are running
<thread 7> is finished. 1 tasks are running
task <thread 8> done
2 tasks are running
<thread 8> is finished. 1 tasks are running
2 tasks are running
task <thread 9> done
<thread 9> is finished. 1 tasks are running
task <thread 5> done
<thread 5> is finished. 0 tasks are running
all tasks done
ok
test_various_ops_small_stack (test.test_threading.ThreadTests.test_various_ops_small_stack) ... with 256 KiB thread stack size...
task <thread 0> will run for 40.6 usec
1 tasks are running
task <thread 1> will run for 24.1 usec
task <thread 0> done
2 tasks are running
task <thread 2> will run for 38.9 usec
task <thread 4> will run for 96.8 usec
task <thread 3> will run for 62.9 usec
task <thread 1> done
task <thread 6> will run for 99.5 usec
task <thread 7> will run for 99.7 usec
task <thread 8> will run for 62.9 usec
task <thread 5> will run for 53.1 usec
task <thread 9> will run for 10.8 usec
<thread 0> is finished. 1 tasks are running
waiting for all tasks to complete
2 tasks are running
<thread 1> is finished. 1 tasks are running
task <thread 2> done
2 tasks are running
3 tasks are running
<thread 2> is finished. 2 tasks are running
task <thread 3> done
task <thread 4> done
3 tasks are running
<thread 3> is finished. 2 tasks are running
task <thread 6> done
3 tasks are running
<thread 4> is finished. 2 tasks are running
task <thread 7> done
<thread 6> is finished. 1 tasks are running
<thread 7> is finished. 0 tasks are running
1 tasks are running
2 tasks are running
task <thread 8> done
<thread 8> is finished. 1 tasks are running
task <thread 5> done
2 tasks are running
task <thread 9> done
<thread 5> is finished. 1 tasks are running
<thread 9> is finished. 0 tasks are running
all tasks done
ok
test_bare_raise_in_brand_new_thread (test.test_threading.ThreadingExceptionTests.test_bare_raise_in_brand_new_thread) ... ok
test_daemonize_active_thread (test.test_threading.ThreadingExceptionTests.test_daemonize_active_thread) ... ok
test_joining_current_thread (test.test_threading.ThreadingExceptionTests.test_joining_current_thread) ... ok
test_joining_inactive_thread (test.test_threading.ThreadingExceptionTests.test_joining_inactive_thread) ... ok
test_multithread_modify_file_noerror (test.test_threading.ThreadingExceptionTests.test_multithread_modify_file_noerror) ... ok
test_print_exception (test.test_threading.ThreadingExceptionTests.test_print_exception) ... ok
test_print_exception_gh_102056 (test.test_threading.ThreadingExceptionTests.test_print_exception_gh_102056) ... ok
test_print_exception_stderr_is_none_1 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_1) ... ok
test_print_exception_stderr_is_none_2 (test.test_threading.ThreadingExceptionTests.test_print_exception_stderr_is_none_2) ... ok
test_recursion_limit (test.test_threading.ThreadingExceptionTests.test_recursion_limit) ... ok
test_releasing_unacquired_lock (test.test_threading.ThreadingExceptionTests.test_releasing_unacquired_lock) ... ok
test_start_thread_again (test.test_threading.ThreadingExceptionTests.test_start_thread_again) ... ok
test_init_immutable_default_args (test.test_threading.TimerTests.test_init_immutable_default_args) ... ok

----------------------------------------------------------------------
Ran 213 tests in 47.094s

OK (skipped=2)
0:00:48 load avg: 0.76 [1/1] test_threading passed in 47.8 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 48.6 sec
Total tests: run=213 skipped=2
Total test files: run=1/1
Result: SUCCESS

localhost:~/development/cpython# ./python -c "import threading; print(hex(threading.current_thread().ident))"
0xf6f0cf34
localhost:~/development/cpython# ./python -m test test_threading
Using random seed: 3415034556
0:00:00 load avg: 0.00 Run 1 test sequentially in a single process
0:00:00 load avg: 0.00 [1/1] test_threading
0:00:46 load avg: 0.47 [1/1] test_threading passed in 46.9 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 46.9 sec
Total tests: run=213 skipped=2
Total test files: run=1/1
Result: SUCCESS
localhost:~/development/cpython#

vfazio requested review from mpage and pitrou April 3, 2025 12:44
pitrou approved these changes Apr 3, 2025
View reviewed changes

pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

+1, LGTM, but we should run some extended CI checks on this

pitrou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 3, 2025

bedevere-bot commented Apr 3, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @pitrou for commit 4ae6d7e 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F130391%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 3, 2025

pitrou commented Apr 3, 2025

Copy link
Copy Markdown
Member

@gpshead Do you think this is a candidate for backporting?

vfazio commented Apr 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@gpshead Do you think this is a candidate for backporting?

I'm secretly hoping so, I plan on porting the patch for Buildroot, but backporting to 3.13 would allow BR to drop it once we hit 3.13.3 hopefully?

pitrou merged commit 7212306 into python:main Apr 4, 2025
pitrou added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Apr 4, 2025

miss-islington-app Bot commented Apr 4, 2025

Copy link
Copy Markdown

Thanks @vfazio for the PR, and @pitrou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington-app Bot commented Apr 4, 2025

Copy link
Copy Markdown

Thanks @vfazio for the PR, and @pitrou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app Bot commented Apr 4, 2025

Copy link
Copy Markdown

Sorry, @vfazio and @pitrou, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 72123063ddee84bb2c9d591a23f420997e35af5a 3.12

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Apr 4, 2025
pythongh-130115: fix thread identifiers for 32-bit musl (pythonGH-130391
57bb72e 
)

CPython's pthread-based thread identifier relies on pthread_t being able
to be represented as an unsigned integer type.

This is true in most Linux libc implementations where it's defined as an
unsigned long, however musl typedefs it as a struct *.

If the pointer has the high bit set and is cast to PyThread_ident_t, the
resultant value can be sign-extended [0]. This can cause issues when
comparing against threading._MainThread's identifier. The main thread's
identifier value is retrieved via _get_main_thread_ident which is backed
by an unsigned long which truncates sign extended bits.

  >>> hex(threading.main_thread().ident)
  '0xb6f33f3c'
  >>> hex(threading.current_thread().ident)
  '0xffffffffb6f33f3c'

Work around this by conditionally compiling in some code for non-glibc
based Linux platforms that are at risk of sign-extension to return a
PyLong based on the main thread's unsigned long thread identifier if the
current thread is the main thread.

[0]: https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Arrays-and-pointers-implementation.html

---------
(cherry picked from commit 7212306)

Co-authored-by: Vincent Fazio <vfazio@gmail.com>
Signed-off-by: Vincent Fazio <vfazio@gmail.com>

bedevere-app Bot commented Apr 4, 2025

Copy link
Copy Markdown

GH-132089 is a backport of this pull request to the 3.13 branch.

bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Apr 4, 2025

pitrou commented Apr 4, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

The threading code base has probably diverged too much from 3.12 to make an automatic backport feasible. You may want to backport an equivalent fix manually @vfazio , or we can limit ourselves to 3.13 if you prefer.

vfazio commented Apr 4, 2025

Copy link
Copy Markdown
Contributor Author

The threading code base has probably diverged too much from 3.12 to make an automatic backport feasible. You may want to backport an equivalent fix manually @vfazio , or we can limit ourselves to 3.13 if you prefer.

Thanks for porting this back! I think back to 3.13 is fine since that's when we first noticed the problem. It was probably an artifact of the change from UL -> ULL (PyThread_ident_t) which only occurred in 3.13+

pitrou pushed a commit that referenced this pull request Apr 4, 2025
[3.13] gh-130115: fix thread identifiers for 32-bit musl (GH-130391) (G…
240c200 
…H-132089)

CPython's pthread-based thread identifier relies on pthread_t being able
to be represented as an unsigned integer type.

This is true in most Linux libc implementations where it's defined as an
unsigned long, however musl typedefs it as a struct *.

If the pointer has the high bit set and is cast to PyThread_ident_t, the
resultant value can be sign-extended [0]. This can cause issues when
comparing against threading._MainThread's identifier. The main thread's
identifier value is retrieved via _get_main_thread_ident which is backed
by an unsigned long which truncates sign extended bits.

  >>> hex(threading.main_thread().ident)
  '0xb6f33f3c'
  >>> hex(threading.current_thread().ident)
  '0xffffffffb6f33f3c'

Work around this by conditionally compiling in some code for non-glibc
based Linux platforms that are at risk of sign-extension to return a
PyLong based on the main thread's unsigned long thread identifier if the
current thread is the main thread.

[0]: https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Arrays-and-pointers-implementation.html

---------
(cherry picked from commit 7212306)

Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Co-authored-by: Vincent Fazio <vfazio@gmail.com>
hugovk removed the needs backport to 3.12 only security fixes label Apr 8, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

pitrou pitrou approved these changes
Yhg1s Awaiting requested review from Yhg1s
mpage Awaiting requested review from mpage

Assignees

pitrou

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

Back | FazBrowse Home | New Git URL