Repository navigation
REPL: AttributeError: module __mp_main__ has no attribute is_prime in ProcessPoolExecutor example #132898
Description
Bug report
Bug description:
python version:3.12.9
An error occurred when running as a standalone script.
An error message is reported when running the sample code of the python document:
Traceback (most recent call last):
File "C:\Users\wen\Desktop\test.py", line 32, in
main()
File "C:\Users\wen\Desktop\test.py", line 28, in main
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures\process.py", line 636, in _chain_from_iterable_of_lists
for element in iterable:
^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Miniconda3\Lib\concurrent\futures_base.py", line 401, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
import concurrent.futures
import math
PRIMES = [
112272535095293,
112582705942171,
112272535095293,
115280095190773,
115797848077099,
1099726899285419]
def is_prime(n):
if n < 2:
return False
if n == 2:
return True
if n % 2 == 0:
return False
sqrt_n = int(math.floor(math.sqrt(n)))
for i in range(3, sqrt_n + 1, 2):
if n % i == 0:
return False
return True
def main():
with concurrent.futures.ProcessPoolExecutor() as executor:
for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
print('%d is prime: %s' % (number, prime))
if __name__ == '__main__':
main()
CPython versions tested on:
3.12
Operating systems tested on:
Windows 10
Linked PRs
- gh-132898: Add a note in multiprocessing.Process docs about creating a process in a REPL session #137118
- [3.14] gh-132898: Add a note in multiprocessing.Process docs about creating a process in a REPL (GH-137118) #137154
- [3.13] gh-132898: Add a note in multiprocessing.Process docs about creating a process in a REPL (GH-137118) #137155
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
picnixz commented on Apr 25, 2025
Question: did this occur because you hit Ctrl+C or killed the process somehow?
Nevermind, it occurred on Linux as well with just running the example.
picnixz commented on Apr 25, 2025
Ok, I only found the issue when the above code is being tested inside the REPL and not as a script (namely python test.py should work but not when I C/C the above code). The best fix for now would be to just indicate in the docs "run it as a standalone script, not inside the REPL", but I don't know the exact reason why (well I can guess: it has something with pickling and probably globals that are not made available because they are only available in the REPL and ProcessPoolExecutor is not able to find the correct "main" module).
cc @pablogsal @gpshead
picnixz commented on Apr 25, 2025
@wenming21 Your latest edit indicates that the error also occurs when running the example as a standalone script. What is the full traceback? (namely, what's above the "Traceback (most recent call last):"? because I do have other stuff written when I encounter the bug)
StanFromIreland commented on Apr 25, 2025
Why was this closed? @wenming21
StanFromIreland commented on Apr 25, 2025
I can also add that this only occurs on the main branch, on 3.13 it works fine.
picnixz commented on Apr 25, 2025
3.14 has seen changes with the default start method so this could be the cause by the way (at least on Linux)
YvesDup commented on Apr 30, 2025
What's happen when you change the start_method ?
OuptputI am on Mac, with a 3.12.8 version and I ran that script:
/bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.py and in REPL, I call main(), I have got errors:
if I changed the start_method to forkserver, I have got the same errors.
And finally if the start method is fork everything is fine.
/usr/local/bin/python3 -i /Users/yves/Desktop/Bugs/gh-132898.py
IMO, this issue is also causing by start_method but I don't figure out how REPL is involved.
And a last remark, if you replace the is_prime function with math.sqrt, the script succeeds in any start_method case .
27 remaining items
Alter chaos multiprocessing start method
Alter chaos multiprocessing start method
Alter chaos multiprocessing start method (#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
fix: move multiprocessing_start_method_fork fixture to shared conftest
Alter chaos multiprocessing start method (RedHatQE#3149)
Alter chaos multiprocessing start method (RedHatQE#3149)