| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
…rectory _has_writeable_tempdir() called tempfile.gettempdir() outside its try/except OSError, so when no temporary directory was usable, import multiprocessing raised FileNotFoundError instead of making spawn the default start method.
…H-159254) gh-155827 made `import multiprocessing` check whether the temporary directory is writeable in order to choose between "forkserver" and "spawn" as the default start method. That check ran at import time and pulled in `multiprocessing.util`, `subprocess`, `tempfile`, `shutil` and `random`, noticeably slowing down the import for every program, including ones that never start a process and every spawned child, the fork server and the resource tracker. The default start method is now determined the first time it is needed instead: on the first `get_context()`, `get_start_method()`, `get_all_start_methods()` or default `Process` start. `multiprocessing.util` goes back to importing `tempfile` only where it is used. Also include the fix from gh-158410 by Vyron Vasileiadis, along with a test adapted from that PR: when `tempfile` finds no usable temporary directory at all, the check now reports that instead of raising `FileNotFoundError`, so that "spawn" is chosen. The regression was reported, and the lazy approach suggested, by Leandro Damascena in #155717 (comment) Co-authored-by: Vyron Vasileiadis <hi@fedonman.com>
|
Thanks! I incorporated this into #159254 and gave you credit there. |
Sorry, something went wrong.
…hod lazily (pythonGH-159254) pythongh-155827 made `import multiprocessing` check whether the temporary directory is writeable in order to choose between "forkserver" and "spawn" as the default start method. That check ran at import time and pulled in `multiprocessing.util`, `subprocess`, `tempfile`, `shutil` and `random`, noticeably slowing down the import for every program, including ones that never start a process and every spawned child, the fork server and the resource tracker. The default start method is now determined the first time it is needed instead: on the first `get_context()`, `get_start_method()`, `get_all_start_methods()` or default `Process` start. `multiprocessing.util` goes back to importing `tempfile` only where it is used. Also include the fix from pythongh-158410 by Vyron Vasileiadis, along with a test adapted from that PR: when `tempfile` finds no usable temporary directory at all, the check now reports that instead of raising `FileNotFoundError`, so that "spawn" is chosen. The regression was reported, and the lazy approach suggested, by Leandro Damascena in python#155717 (comment) (cherry picked from commit 48ddda1) Co-authored-by: Vyron Vasileiadis <hi@fedonman.com>
…zily (GH-159254) (GH-159271) gh-155827 made `import multiprocessing` check whether the temporary directory is writeable in order to choose between "forkserver" and "spawn" as the default start method. That check ran at import time and pulled in `multiprocessing.util`, `subprocess`, `tempfile`, `shutil` and `random`, noticeably slowing down the import for every program, including ones that never start a process and every spawned child, the fork server and the resource tracker. The default start method is now determined the first time it is needed instead: on the first `get_context()`, `get_start_method()`, `get_all_start_methods()` or default `Process` start. `multiprocessing.util` goes back to importing `tempfile` only where it is used. Also include the fix from gh-158410 by Vyron Vasileiadis, along with a test adapted from that PR: when `tempfile` finds no usable temporary directory at all, the check now reports that instead of raising `FileNotFoundError`, so that "spawn" is chosen. The regression was reported, and the lazy approach suggested, by Leandro Damascena in #155717 (comment) (cherry picked from commit 48ddda1) Co-authored-by: Vyron Vasileiadis <hi@fedonman.com> * gh-155717: Reword the NEWS entry for the default start method change Say that the change applies when there is no writable temporary directory and that starting a process previously failed there. --------- Co-authored-by: Vyron Vasileiadis <hi@fedonman.com>
…ly filesystems (GH-155827) (#155886) * gh-155717: use `spawn` as the default start method for read-only filesystems (GH-155827) The "forkserver" start method (the default start method on non-Windows systems) requires the ability to write temporary files, which is not possible if TMPDIR is read-only (e.g., k8s containers mounted with `readOnlyRootFilesystem=True`). On such filesystems, the default start method changes from "forkserver" to "spawn". (cherry picked from commit c144799) Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> * Update Lib/test/_test_multiprocessing.py * [3.14] gh-155717: Determine multiprocessing's default start method lazily (GH-159254) gh-155827 made `import multiprocessing` check whether the temporary directory is writeable in order to choose between "forkserver" and "spawn" as the default start method. That check ran at import time and pulled in `multiprocessing.util`, `subprocess`, `tempfile`, `shutil` and `random`, noticeably slowing down the import for every program, including ones that never start a process and every spawned child, the fork server and the resource tracker. The default start method is now determined the first time it is needed instead: on the first `get_context()`, `get_start_method()`, `get_all_start_methods()` or default `Process` start. `multiprocessing.util` goes back to importing `tempfile` only where it is used. Also include the fix from gh-158410 by Vyron Vasileiadis, along with a test adapted from that PR: when `tempfile` finds no usable temporary directory at all, the check now reports that instead of raising `FileNotFoundError`, so that "spawn" is chosen. The regression was reported, and the lazy approach suggested, by Leandro Damascena in #155717 (comment) (cherry picked from commit 48ddda1) Co-authored-by: Vyron Vasileiadis <hi@fedonman.com> * gh-155717: Reword the NEWS entry for the default start method change Say that the change applies when there is no writable temporary directory and that starting a process previously failed there. --------- Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com> Co-authored-by: Vyron Vasileiadis <hi@fedonman.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
| Back | FazBrowse Home | New Git URL |
When tempfile finds no usable temporary directory, which is the setup this issue is about, import multiprocessing on main raises FileNotFoundError ((ulimit -f 0; ./python -c 'import multiprocessing') shows it), because _has_writeable_tempdir() calls tempfile.gettempdir() outside its try. This moves the call inside it, so the import succeeds and the default start method is spawn.
No NEWS entry, since GH-155827 has not been released yet.