fix(extension_testcapi): Locate python3.lib on single-config generators - #453
Conversation
Sorry, something went wrong.
Linking _testcapi failed on Windows with the Ninja and NMake generators:
LINK : fatal error LNK1104: cannot open file 'python3.lib'
_testcapi contains translation units that define Py_LIMITED_API
(vectorcall_limited.c, heaptype_relative.c). PC/pyconfig.h then emits
`pragma comment(lib,"python3.lib")`, so the stable-ABI import library
must be on the linker's search path.
The search path was built by hand as
${PROJECT_BINARY_DIR}/CMakeBuild/libpython/$<CONFIG>
$<CONFIG> expands for every generator, but only multi-config generators
append the configuration to a target's archive output directory. Since
libpython3-shared sets LIBRARY_OUTPUT_DIRECTORY and
RUNTIME_OUTPUT_DIRECTORY but no ARCHIVE_OUTPUT_DIRECTORY, on a
single-config generator python3.lib lands directly in the libpython
binary directory -- one level above where the $<CONFIG> form looked.
Visual Studio agreed with the hardcoded path, which is why this went
unnoticed.
Use $<TARGET_LINKER_FILE_DIR:libpython3-shared> so the directory comes
from the target itself and stays correct for both generator classes.
The target is already a hard dependency of extension_testcapi on the
next line, so it is guaranteed to exist here.
|
Verified on both generator classes. Branch under test: e815ed7989aa9b544d153d4c80199ec0b121fe82. Environment: Windows 11, MSVC 19.38.33130 (VS 2022 17.8), CMake 4.4.2, Python 3.12.10, Release. Both builds consume this branch through 3D Slicer's SuperBuild via -DSlicer_python_GIT_REPOSITORY=... -DSlicer_python_GIT_TAG=fix-testcapi-python3lib-single-config. Single-config (Ninja) — the failure this fixesBefore, at [721/737]: FAILED: Lib/lib-dynload/_testcapi.pyd libs/_testcapi.lib LINK : fatal error LNK1104: cannot open file 'python3.lib' After: 288256 python-build/Lib/lib-dynload/_testcapi.pyd 1766 python-build/libs/_testcapi.lib Multi-config (Visual Studio 17 2022, x64) — the regression checkCMAKE_CONFIGURATION_TYPES=Debug;Release;MinSizeRel;RelWithDebInfo extension_testcapi.vcxproj -> C:\...\python-build\Lib\lib-dynload\Release\_testcapi.pyd -- Installing: C:/.../python-install/libs/_testcapi.lib -- Installing: C:/.../python-install/Lib/lib-dynload/_testcapi.pyd 287744 python-build/Lib/lib-dynload/Release/_testcapi.pyd 1766 python-build/libs/Release/_testcapi.lib No link errors in either build. Why the change is the right shapeThe two builds put the import library in different places, and $<TARGET_LINKER_FILE_DIR:libpython3-shared> follows it to both:
That is the whole defect in one table: the hardcoded .../CMakeBuild/libpython/$<CONFIG> matched the second row only, because $<CONFIG> expands under every generator while only multi-config generators add the configuration directory to a target's archive output. Asking the target removes the assumption instead of re-encoding it. Happy to add a CI job covering a single-config Windows generator if that would be useful — the gap is that _testcapi is only ever linked under Visual Studio today. |
Sorry, something went wrong.
Fixes #452.
Problem
Linking _testcapi fails on Windows with single-config generators (Ninja, NMake Makefiles):
Multi-config generators (Visual Studio) are unaffected.
_testcapi has translation units compiled with Py_LIMITED_API, so PC/pyconfig.h emits pragma comment(lib,"python3.lib") and the stable-ABI import library must be on the linker search path. That path was reconstructed by hand:
$<CONFIG> expands for every generator, but only multi-config generators append the configuration to a target's archive output directory. libpython3-shared sets LIBRARY_OUTPUT_DIRECTORY and RUNTIME_OUTPUT_DIRECTORY but no ARCHIVE_OUTPUT_DIRECTORY, so under Ninja python3.lib is written straight to CMakeBuild/libpython/ — one level above where the $<CONFIG> form looked.
Change
The directory now comes from the target itself, so it is correct for both generator classes and cannot drift if the output location changes later. libpython3-shared is already a hard dependency of extension_testcapi on the next line, so it is guaranteed to exist here.
No behaviour change for multi-config generators: $<TARGET_LINKER_FILE_DIR:libpython3-shared> resolves to the same per-configuration directory the hardcoded path produced.
Verification
Windows 11, MSVC 19.38 (VS 2022 17.8), CMake 4.4.2, Ninja, Python 3.12.10, CMAKE_BUILD_TYPE=Release.
Before: _testcapi.pyd fails with LNK1104, aborting the build at ~721/737.
After: _testcapi.pyd links and installs (Lib/lib-dynload/_testcapi.pyd, libs/_testcapi.lib), and the build proceeds.
Found while building 3D Slicer's SuperBuild, which vendors this project and must use Ninja because it is the only generator honouring CMAKE_<LANG>_COMPILER_LAUNCHER (ccache).