FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

gh-139462: Make the ProcessPoolExecutor BrokenProcessPool exception report which child process terminated by jberg5 · Pull Request #139486 · python/cpython · GitHub

Repository navigation

10 changes: 10 additions & 0 deletions Doc/whatsnew/3.15.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -369,6 +369,16 @@ collections.abc
:mod:`!collections.abc` module.


concurrent.futures
------------------

* Improved error reporting when a child process in a
:class:`concurrent.futures.ProcessPoolExecutor` terminates abruptly.
The resulting traceback will now tell you the PID and exit code of the
terminated process.
(Contributed by Jonathan Berg in :gh:`139486`.)


dataclasses
-----------

Expand Down
18 changes: 16 additions & 2 deletions Lib/concurrent/futures/process.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -474,9 +474,23 @@ def _terminate_broken(self, cause):
bpe = BrokenProcessPool("A process in the process pool was "
"terminated abruptly while the future was "
"running or pending.")
cause_str = None
if cause is not None:
bpe.__cause__ = _RemoteTraceback(
f"\n'''\n{''.join(cause)}'''")
cause_str = ''.join(cause)
else:
# No cause known, so report any processes that have
# terminated with nonzero exit codes, e.g. from a
# segfault. Multiple may terminate simultaneously,
# so include all of them in the traceback.
errors = []
Comment thread
jberg5 marked this conversation as resolved.
for p in self.processes.values():
if p.exitcode is not None and p.exitcode != 0:
errors.append(f"Process {p.pid} terminated abruptly "
f"with exit code {p.exitcode}")
if errors:
cause_str = "\n".join(errors)
if cause_str:
bpe.__cause__ = _RemoteTraceback(f"\n'''\n{cause_str}'''")

# Mark pending tasks as failed.
for work_id, work_item in self.pending_work_items.items():
Expand Down
15 changes: 15 additions & 0 deletions Lib/test/test_concurrent_futures/test_process_pool.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,21 @@ def test_traceback(self):
self.assertIn('raise RuntimeError(123) # some comment',
f1.getvalue())

def test_traceback_when_child_process_terminates_abruptly(self):
# gh-139462 enhancement - BrokenProcessPool exceptions
# should describe which process terminated.
exit_code = 99
with self.executor_type(max_workers=1) as executor:
future = executor.submit(os._exit, exit_code)
with self.assertRaises(BrokenProcessPool) as bpe:
future.result()

cause = bpe.exception.__cause__
self.assertIsInstance(cause, futures.process._RemoteTraceback)
self.assertIn(
f"terminated abruptly with exit code {exit_code}", cause.tb
)

@warnings_helper.ignore_fork_in_thread_deprecation_warnings()
@hashlib_helper.requires_hashdigest('md5')
def test_ressources_gced_in_workers(self):
Expand Down
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
When a child process in a :class:`concurrent.futures.ProcessPoolExecutor`
terminates abruptly, the resulting traceback will now tell you the PID
and exit code of the terminated process. Contributed by Jonathan Berg.
Loading

Back | FazBrowse Home | New Git URL