FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows by brianquinlan · Pull Request #13132 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows - #13132

Merged
zooba merged 3 commits into
python:masterfrom
brianquinlan:issue26903
May 8, 2019
ConversationCommits (3)ChecksFiles changed
Merged

bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows#13132
zooba merged 3 commits into
python:masterfrom
brianquinlan:issue26903

Conversation

brianquinlan commented May 6, 2019 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Contributor

zooba approved these changes May 6, 2019
View reviewed changes
Comment thread Lib/test/test_concurrent_futures.py Outdated
Update Lib/test/test_concurrent_futures.py
97c1084 
Co-Authored-By: brianquinlan <brian@sweetapp.com>
JelleZijlstra reviewed May 7, 2019
View reviewed changes
Comment thread Lib/concurrent/futures/process.py
raise ValueError("max_workers must be greater than 0")
elif (sys.platform == 'win32' and
max_workers > _MAX_WINDOWS_WORKERS):
raise ValueError(

JelleZijlstra May 7, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Should we consider automatically changing max_workers to 61 instead? That would make for a nicer API for people using max_workers=os.cpu_count(), and it would make code like what was in Black before https://github.com/python/black/pull/838/files work unchanged.

On the other hand, it may be unexpected that we silently use fewer workers than the user asked for.

brianquinlan May 7, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I thought silently using fewer processes could be surprising. I also think that it's pretty unusual in the standard library to silently fix bad argument values.

zooba May 8, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I like raising when too many workers are requested, and doing it quietly when it was "automatic".

zooba merged commit 3988986 into python:master May 8, 2019

miss-islington commented May 8, 2019

Copy link
Copy Markdown
Contributor

Thanks @brianquinlan for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

zooba commented May 8, 2019

Copy link
Copy Markdown
Member

Thanks, Brian!

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 8, 2019
bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (python…
c805d53 
…GH-13132)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>

bedevere-bot commented May 8, 2019

Copy link
Copy Markdown

GH-13206 is a backport of this pull request to the 3.7 branch.

miss-islington commented May 29, 2019

Copy link
Copy Markdown
Contributor

Thanks @brianquinlan for the PR, and @zooba for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 29, 2019
bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (python…
4ad2b16 
…GH-13132)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>

bedevere-bot commented May 29, 2019

Copy link
Copy Markdown

GH-13643 is a backport of this pull request to the 3.7 branch.

ned-deily pushed a commit that referenced this pull request May 29, 2019
bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132…
8ea0fd8 
…) (GH-13643)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
mingwandroid pushed a commit to mingwandroid/cpython that referenced this pull request Sep 3, 2020
bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (python…
24f3738 
…GH-13132) (pythonGH-13643)

Co-Authored-By: brianquinlan <brian@sweetapp.com>
(cherry picked from commit 3988986)

Co-authored-by: Brian Quinlan <brian@sweetapp.com>
saschanaz mentioned this pull request Sep 1, 2021
Closed
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
Fix EM_PYTHON_MULTIPROCESSING under windows
4e73a3d 
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 mentioned this pull request Sep 20, 2022
Merged
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
Fix EM_PYTHON_MULTIPROCESSING under windows
1676673 
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
Fix EM_PYTHON_MULTIPROCESSING under windows
f925fc2 
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
sbc100 added a commit to emscripten-core/emscripten that referenced this pull request Sep 20, 2022
Fix EM_PYTHON_MULTIPROCESSING under windows (#17892)
8494ac4 
This mirrors the upstream bugfix in python 3.8.0 which limits the
size of the worker pool to 61: python/cpython#13132

Fixes: #13785
diogocp mannequin mentioned this pull request Oct 21, 2023
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

JelleZijlstra JelleZijlstra left review comments
zooba zooba approved these changes

Assignees

zooba

Labels

OS-windows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

Back | FazBrowse Home | New Git URL