gh-107219: Fix concurrent.futures terminate_broken() - #108974
Merged
vstinner merged 1 commit intoSep 6, 2023
Merged
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
Fix a race condition in _ExecutorManagerThread.terminate_broken(): ignore the InvalidStateError on future.set_exception(). It can happen if the future is cancelled before the caller. Moreover, test_crash_big_data() now waits explicitly until the executor completes.
Copy link
Copy Markdown
Member
Author
|
This change doesn't fix the most annoying bug of issue #107219. test_crash_big_data() still hangs on Windows. But it fix this error: test_crash_big_data (test.test_concurrent_futures.ProcessPoolSpawnExecutorDeadlockTest.test_crash_big_data) ... Warning -- Uncaugh
t thread exception: InvalidStateError
Exception in thread Thread-9:
Traceback (most recent call last):
File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\threading.py", line 1059, in _bootstrap_inner
self.run()
File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\concurrent\futures\process.py", line 344, in run
self.terminate_broken(cause)
File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\concurrent\futures\process.py", line 492, in terminate_broken
work_item.future.set_exception(bpe)
File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\concurrent\futures\_base.py", line 559, in set_exception
raise InvalidStateError('{}: {!r}'.format(self._state, self))
concurrent.futures._base.InvalidStateError: CANCELLED: <Future at 0x2cf561cf260 state=cancelled>
0.55s Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
Warning -- Dangling thread: <_MainThread(MainThread, started 11880)>
Warning -- Dangling thread: <Thread(QueueFeederThread, started daemon 4112)>
ok
link: #107219 (comment) |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
Author
If it wasn't obvious enough, the test decided to hang on Windows x64 CI :-) 0:41:15 load avg: 0.02 [462/462/3] test.test_concurrent_futures.test_deadlock process crashed (Exit code 1) Timeout (0:20:00)! Thread 0x00000c90 (most recent call first): File "D:\a\cpython\cpython\Lib\multiprocessing\connection.py", line 282 in _send_bytes File "D:\a\cpython\cpython\Lib\multiprocessing\connection.py", line 199 in send_bytes File "D:\a\cpython\cpython\Lib\multiprocessing\queues.py", line 246 in _feed File "D:\a\cpython\cpython\Lib\threading.py", line 996 in run File "D:\a\cpython\cpython\Lib\threading.py", line 1059 in _bootstrap_inner File "D:\a\cpython\cpython\Lib\threading.py", line 1016 in _bootstrap Thread 0x00001b78 (most recent call first): File "D:\a\cpython\cpython\Lib\threading.py", line 1153 in _wait_for_tstate_lock File "D:\a\cpython\cpython\Lib\threading.py", line 1133 in join File "D:\a\cpython\cpython\Lib\multiprocessing\queues.py", line 199 in _finalize_join File "D:\a\cpython\cpython\Lib\multiprocessing\util.py", line 224 in __call__ File "D:\a\cpython\cpython\Lib\multiprocessing\queues.py", line 151 in join_thread File "D:\a\cpython\cpython\Lib\concurrent\futures\process.py", line 560 in join_executor_internals File "D:\a\cpython\cpython\Lib\concurrent\futures\process.py", line 514 in terminate_broken File "D:\a\cpython\cpython\Lib\concurrent\futures\process.py", line 344 in run File "D:\a\cpython\cpython\Lib\threading.py", line 1059 in _bootstrap_inner File "D:\a\cpython\cpython\Lib\threading.py", line 1016 in _bootstrap Thread 0x000012cc (most recent call first): File "D:\a\cpython\cpython\Lib\threading.py", line 1153 in _wait_for_tstate_lock File "D:\a\cpython\cpython\Lib\threading.py", line 1133 in join File "D:\a\cpython\cpython\Lib\concurrent\futures\process.py", line 843 in shutdown File "D:\a\cpython\cpython\Lib\concurrent\futures\_base.py", line 647 in __exit__ File "D:\a\cpython\cpython\Lib\test\test_concurrent_futures\test_deadlock.py", line 236 in test_crash_big_data |
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix a race condition in _ExecutorManagerThread.terminate_broken(): ignore the InvalidStateError on future.set_exception(). It can happen if the future is cancelled before the caller.
Moreover, test_crash_big_data() now waits explicitly until the executor completes.