Repository navigation
Tests fail when _interpreters extension is disabled #148244
Description
Bug report
Bug description:
My CPython version: 3.14.4 (problem with 3.14.3 too)
Configure:
./configure --disable-shared --enable-optimizations --without-lto
Select modules:
./Modules/makesetup -c ./Modules/config.c.in -s ./Modules ./Setup.my.local mv config.c ./Modules/
File (github says *.local file upload is not expected, change ext txt -> local)
Setup.my.txt
As you can see, the _interpreters module is disabled
And make:
make -j4
Error after testing:
...
File ".../Lib/test/datetimetester.py", line 7304, in test_concurrent_initialization_subinterpreter
...
Traceback (most recent call last):
File "<string>", line 2, in <module>
from concurrent.futures import InterpreterPoolExecutor
ImportError: cannot import name 'InterpreterPoolExecutor' from 'concurrent.futures'
...
Ran 1098 tests in 10.101s
FAILED (failures=1, skipped=53)
test test_datetime failed
make: *** [Makefile:990: profile-run-stamp] Error 2
I think a test with a non-existent module should be skipped, not crashed
Also, why does a test failure cause the build to fail if these tests are just needed to collect statistics?
CPython versions tested on:
3.14
Operating systems tested on:
Linux
Activity
StanFromIreland commented on Apr 8, 2026
The build succeed, just the test failed.
TrueCat17 commented on Apr 8, 2026
No, building stops after tests failed (in my case)
Shrey-N commented on Apr 8, 2026
Hiya, I believe that the failure is caused by InterpreterPoolExecutor being missing from concurrent.futures when the _interpreters module is disabled, The test should be skipped in this case
Adding @unittest.skipIf(_interpreters is None, "requires _interpreters") "requires _interpreters") to the test resolves this issue
I am just making a PR fixing this issue within a minute!
Thank you for the bug report :)
StanFromIreland commented on Apr 8, 2026
@sunmy2019 out of curiosity, what does this have to do with building?
Fixes PGO building failed due to tests python#148244
Shrey-N commented on Apr 8, 2026
I think the build stops because the Makefile is configured to quit immediately if any test crashes.
vstinner commented on Apr 8, 2026
Currently, we don't support custom Python builds disabling the _interpreters extension. If you do that, you have to patch the test suite to skip tests using it.
StanFromIreland commented on Apr 8, 2026
But that's the same for any test...? It has nothing to with building in particular.
StanFromIreland commented on Apr 8, 2026
Disabling _interpreters will lead to many other fails, on my system I get:
14 tests failed: test.test_concurrent_futures.test_as_completed test.test_concurrent_futures.test_deadlock test.test_concurrent_futures.test_future test.test_concurrent_futures.test_init test.test_concurrent_futures.test_interpreter_pool test.test_concurrent_futures.test_process_pool test.test_concurrent_futures.test_shutdown test.test_concurrent_futures.test_thread_pool test.test_concurrent_futures.test_wait test__interpchannels test_datetime test_interpretersShrey-N commented on Apr 8, 2026
Oh, yea you are right, _interpreters being disabled breaks many tests I will update the fix, i think we can add skips to the test_interpreters package and the executor tests in test_concurrent_futures
Shrey-N commented on Apr 8, 2026
Got it @vstinner, Just to clarify does that mean a patch adding these test skips wouldn't be accepted? I think it would be a nice thing to add
1 remaining item
sunmy2019 commented on Apr 8, 2026
PGO requires build twice:
Stage 1. Normal Build
Stage 2. Collect Profile Data (Here, we run a python test.)
Stage 3. Build Again with Profile Data.
I mean, the stage3 fails to proceed becauses stage2 was too strict. Usually some errors are allowed in stage2.
StanFromIreland commented on Apr 8, 2026
The build label is for changes to the build process, here it’s working correctly, the issue is that tests don’t handle a missing _interpreters module.
TrueCat17 commented on Apr 8, 2026
As I already wrote, it seems to me that the problem is that the build system generally pays some attention to the results of the tests, while these tests themselves are needed only for collecting statistics
Am I wrong?
TrueCat17 commented on Apr 8, 2026
I solved the problem
In the Makefile.pre.in file (before the configuration stage), in the profile-run-stamp label, in the end of line $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK), you need to add || true
Because of this, test results during the PGO build phase will be ignored, which is what they should be doing, in my opinion
I'm not sure if this is the best possible solution, but it seems to be working for me
TrueCat17 commented on Apr 29, 2026
Take the time to figure out when this behavior started
The answer is after this PR: 6ab6040
@vstinner As I understand it, it was decided to do this to fix tests more quickly, but is it possible to add some parameter to ./configure so that if it is specified, failure of tests will not lead to failure of the build?
For example: ./configure --enable-optimizations --ignore-profile-test-failure
Maybe need to replace the past || true -> || $ignore_test_failure, when variable ignore_test_failure should initialized to false, and if there is a special change the parameter to true
Or maybe it’s the other way around, initially true, and if there is a some parameter specified by the building in CI, then set it to false
vstinner commented on Apr 29, 2026
I don't think that it's a good idea to ignore failures of the $PROFILE_TASK. It can produce less efficient Python executable.
If you consider that Python should support build without the _interpreters extension, please open a discussion at https://discuss.python.org/. Currently, it's not supported and you didn't explain why you disable this extension.
TrueCat17 commented on Apr 29, 2026
I didn't disable this extension, I just didn't enable it
When I saw the error, I tried to enable it, but I got a big bunch of other errors
I don’t know what the reason is, but (for me) adding || true is much easier than fixing almost a dozen tests
And I don’t think that there is a decrease in performance, because the number of failed tests is much less than the number of successful ones
The problem is not in specific tests, but in the fact that any, even the smallest error in the tests, stops the PGO build, although the tests there are needed only as a “code filler”, which, in principle, should not be able to stop the build
vstinner commented on Apr 29, 2026
By default, the _interpreters extension is built. You customized the build with a custom Setup.my.local file. You should enable _interpreters in this file, or at least not disable it.
Why do you need a Setup.my.local file? How did you create this file?
vstinner commented on Apr 29, 2026
You used ./Modules/makesetup -c ./Modules/config.c.in -s ./Modules ./Setup.my.local to generate config.c. The Makefile default command is different, it uses more Setup files:
If you don't want to use Modules/Setup.stdlib, Modules/Setup.bootstrap, and Modules/Setup, you should add _interpreters _interpretersmodule.c to your Setup file.
TrueCat17 commented on Apr 29, 2026
I'll see later if specifying these additional files will improve my situation
What about the main part?
Do you disagree with me?
vstinner commented on Apr 29, 2026
I was the one who modified the Python PGO build to no longer ignore errors in tests (in PROFILE_TASK). So no, I disagree with you. Ignoring silently tests errors is a bad idea. As I wrote previously, it can lead to a less efficient Python executable. The PROFILE_TASK is very important for PGO.
See: https://docs.python.org/dev/using/configure.html#performance-options.
TrueCat17 commented on Apr 29, 2026
In my case, PGO gives about 10% performance and a reduction in the weight of the executable file by 0.5 MB, despite ignoring errors in tests
According to you, I should either abandon PGO, or fix all the tests that stopped working due to some missing modules, or enable all these modules (that I don’t need), increasing the size of the final executable files (and I have 3 of them for different platforms, and they are distributed in 1 archive, so if unnecessary modules add 1 MB, then in my case 3 MB will be added, i.e. somewhere + 20% for nothing)
As a result, all these 3 solutions are bad, and it’s easier for me to just add || true at the end of one line, rather than bother with all this
That’s why I proposed making a special parameter, which, when enabled, would return ignoring test failures
You might be thinking: If tests fail, it could lead to poor performance.
I think: if a PGO build stops due to tests, then a normal build will be used, where performance loss is guaranteed
vstinner commented on Apr 29, 2026
So your motivation to disable extensions is to reduce the Python installation size? On my Fedora 43, /usr/lib64/python3.15/lib-dynload/_interpreters.cpython-315-x86_64-linux-gnu.so is only 44 KB. Is it really worth it to omit the _interpreters extension?
As I already wrote, if you consider that omitting the _interpreters extension is worth it, please open a discussion at https://discuss.python.org/ to explain your use case.
TrueCat17 commented on Apr 29, 2026
As I already said, this extension is just one of those that is required for all tests to work successfully, and in addition, all these required extensions have other extensions as dependencies, as a result, the whole thing grows like a snowball