| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
… testing is INCOMPLETE.
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Sorry, something went wrong.
|
/ok to test |
Sorry, something went wrong.
…test_load_nvidia_dynamic_lib.py
…or conda anomaly, to restore proper functioning for standard CTK installations
|
/ok to test |
Sorry, something went wrong.
Manually tested with: pip install nvidia-cufftmp-cu13==12.1.3.1 That wheel was yanked, therefore not adding to pyproject.toml
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Sorry, something went wrong.
|
/ok to test |
Sorry, something went wrong.
There was a problem hiding this comment.
sequenceDiagram
participant User
participant find_nvidia_header_directory
participant find_nvidia_headers.py
participant supported_nvidia_headers.py
participant load_nvidia_dynamic_lib
participant supported_nvidia_libs.py
participant Test Suite
User->>find_nvidia_header_directory: "Request header for libname (e.g., 'cutensor')"
find_nvidia_header_directory->>supported_nvidia_headers.py: "Check if libname in SUPPORTED_HEADERS_CTK"
alt CTK Library
supported_nvidia_headers.py-->>find_nvidia_header_directory: "Found in CTK"
find_nvidia_header_directory->>find_nvidia_headers.py: "Call _find_ctk_header_directory()"
else Non-CTK Library (e.g., cutensor)
supported_nvidia_headers.py-->>find_nvidia_header_directory: "Check SUPPORTED_HEADERS_NON_CTK"
find_nvidia_header_directory->>supported_nvidia_headers.py: "Get canonical header basename"
supported_nvidia_headers.py-->>find_nvidia_header_directory: "Return 'cutensor.h'"
find_nvidia_header_directory->>find_nvidia_headers.py: "Search site-packages"
find_nvidia_headers.py->>find_nvidia_headers.py: "Check SUPPORTED_SITE_PACKAGE_HEADER_DIRS_NON_CTK"
find_nvidia_headers.py->>find_nvidia_headers.py: "_find_based_on_conda_layout() for non-CTK"
alt Windows Conda Anomaly (cccl)
find_nvidia_headers.py->>find_nvidia_headers.py: "Handle targets/x64 path anomaly"
end
find_nvidia_headers.py->>find_nvidia_headers.py: "Check SUPPORTED_INSTALL_DIRS_NON_CTK"
end
find_nvidia_header_directory-->>User: "Return header directory path or None"
User->>load_nvidia_dynamic_lib: "Load dynamic library (e.g., 'cutensor')"
load_nvidia_dynamic_lib->>supported_nvidia_libs.py: "Query SUPPORTED_LINUX_SONAMES or SUPPORTED_WINDOWS_DLLS"
alt CTK Library
supported_nvidia_libs.py-->>load_nvidia_dynamic_lib: "Return from SUPPORTED_*_SONAMES_CTK"
else Non-CTK Library (cutensor)
supported_nvidia_libs.py-->>load_nvidia_dynamic_lib: "Return from SUPPORTED_*_SONAMES_OTHER"
Note over supported_nvidia_libs.py: "Added: cutensor -> libcutensor.so.2 (Linux)<br/>cutensor -> cutensor.dll (Windows)"
end
load_nvidia_dynamic_lib->>supported_nvidia_libs.py: "Check SITE_PACKAGES_LIBDIRS_*"
supported_nvidia_libs.py-->>load_nvidia_dynamic_lib: "Return site-packages paths"
load_nvidia_dynamic_lib-->>User: "Return LoadedDL object or DynamicLibNotFoundError"
User->>Test Suite: "Run test_find_nvidia_headers"
Test Suite->>find_nvidia_header_directory: "Test all SUPPORTED_HEADERS_NON_CTK.keys()"
Test Suite->>Test Suite: "Loop over cutensor, nvshmem, etc."
Test Suite-->>User: "Report test results"
User->>Test Suite: "Run test_load_nvidia_dynamic_lib"
Test Suite->>load_nvidia_dynamic_lib: "Test all SUPPORTED_LINUX_SONAMES or SUPPORTED_WINDOWS_DLLS"
Test Suite->>Test Suite: "Run in spawned child process for isolation"
Test Suite-->>User: "Report test results with abs_path or 'Not found'"
14 files reviewed, 12 comments
Sorry, something went wrong.
|
@ZzEeKkAa I believe this is ready for review. It'd be great if you could take a look. The only thing left to do is go through the manual testing systematically. |
Sorry, something went wrong.
|
CI ran successfully at commit b079c84: https://github.com/NVIDIA/cuda-python/actions/runs/18886735456/job/53903916554?pr=1194 |
Sorry, something went wrong.
There was a problem hiding this comment.
This incremental review covers only the changes made since the last review, not the entire PR. The developer has addressed previous feedback by applying the _abs_norm() wrapper to all remaining return paths in find_nvidia_header_directory() (lines 150, 153, 159) and fixed the PowerShell syntax error in the conda setup script. The _abs_norm() helper (lines 15-18) normalizes path separators and converts relative paths to absolute paths, ensuring consistent path format across all return points—critical for the new cuTENSOR/non-CTK library support that can be installed in diverse locations (site-packages, conda, standard directories). The PowerShell script now includes 'cutensor' in the package list for testing and corrects the missing comma after "libnvshmem-dev". These changes ensure that header discovery returns predictable, normalized paths regardless of installation method or platform, addressing Windows conda path anomalies mentioned in the PR description.
| Filename | Score | Overview |
|---|---|---|
| cuda_pathfinder/cuda/pathfinder/_headers/find_nvidia_headers.py | 5/5 | Applied _abs_norm() wrapper to all non-CTK header return paths for consistent path normalization |
| toolshed/conda_create_for_pathfinder_testing.ps1 | 5/5 | Added 'cutensor' package and fixed trailing comma syntax error in package list |
2 files reviewed, no comments
Sorry, something went wrong.
|
/ok to test |
Sorry, something went wrong.
…headers.py (mirrors supported_nvidia_libs.py)
|
I'm through one pass of running all 18 combinations of manual tests. I'll log the results here tomorrow. |
Sorry, something went wrong.
|
Manual QA, based on local tests following this scheme: Results based on the logs in the attached pr1194_local_testing_pass01_2025-10-29+094920.zip local-ctk headers (note that these are the only matches for the grep) pr1194_local_testing/pass01 $ grep 'hdr_dir=None' *.txt | sort -t'[' -k2 2lc_1lx64_cu12.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_1lx64_cu13.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_2la64_cu12.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_2la64_cu13.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_3wx64_cu12.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_3wx64_cu13.txt:INFO test_find_non_ctk_headers[cutensor]: hdr_dir=None 2lc_2la64_cu12.txt:INFO test_find_non_ctk_headers[nvshmem]: hdr_dir=None 2lc_2la64_cu13.txt:INFO test_find_non_ctk_headers[nvshmem]: hdr_dir=None Explanation: cutensor and nvshmem are not installed when testing with local-ctk. site-packages libs pr1194_local_testing/pass01 $ grep 'Not found:' 1sp_*.txt | sort -t"'" -k2 1sp_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 1sp_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 1sp_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 1sp_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 1sp_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 1sp_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 1sp_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' Explanations:
conda-ctk libs pr1194_local_testing/pass01 $ grep 'Not found:' 3cc_*.txt | sort -t"'" -k2 3cc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 3cc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 3cc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 3cc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' Explanations:
local-ctk libs pr1194_local_testing/pass01 $ grep 'Not found:' 2lc_*.txt | sort -t"'" -k2 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cublasmp]: Not found: libname='cublasmp' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cublasmp]: Not found: libname='cublasmp' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cublasmp]: Not found: libname='cublasmp' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cublasmp]: Not found: libname='cublasmp' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cudss]: Not found: libname='cudss' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensor]: Not found: libname='cutensor' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cutensorMg]: Not found: libname='cutensorMg' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nccl]: Not found: libname='nccl' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nccl]: Not found: libname='nccl' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nccl]: Not found: libname='nccl' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nccl]: Not found: libname='nccl' 2lc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 2lc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nvpl_fftw]: Not found: libname='nvpl_fftw' 2lc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[nvshmem_host]: Not found: libname='nvshmem_host' 2lc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[nvshmem_host]: Not found: libname='nvshmem_host' Explanation: These are all non-CTK libs; only the local-ctk is installed. A special look at cufftMp: pr1194_local_testing/pass01 $ grep 'INFO test_load_nvidia_dynamic_lib\[cufftMp\]' 1sp_*.txt 3cc_*.txt 1sp_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: abs_path=/home/rgrossekunst/forked/cuda-python/SpCu12Venv/lib/python3.12/site-packages/nvidia/cufftmp/cu12/lib/libcufftMp.so.11 1sp_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 1sp_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: abs_path=/wrk/forked/cuda-python/SpCu12Venv/lib/python3.12/site-packages/nvidia/cufftmp/cu12/lib/libcufftMp.so.11 1sp_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 3cc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: abs_path=/home/rgrossekunst/miniforge3/envs/pathfinder_cu12.9.1/lib/libcufftMp.so 3cc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' 3cc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: abs_path=/wrk/miniforge3/envs/pathfinder_testing_cu12.9.1/lib/libcufftMp.so 3cc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[cufftMp]: Not found: libname='cufftMp' Confirms the explanations above. A special look at mathdx: pr1194_local_testing/pass01 $ grep 'INFO test_load_nvidia_dynamic_lib\[mathdx\]' 1sp_*.txt 3cc_*.txt 1sp_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/home/rgrossekunst/forked/cuda-python/SpCu12Venv/lib/python3.12/site-packages/nvidia/cu12/lib/libmathdx.so.0 1sp_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 1sp_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/wrk/forked/cuda-python/SpCu12Venv/lib/python3.12/site-packages/nvidia/cu12/lib/libmathdx.so.0 1sp_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 1sp_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=C:\Users\rgrossekunst\forked\cuda-python\Sp12Venv\Lib\site-packages\nvidia/cu12/bin\mathdx64_0.dll 1sp_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: Not found: libname='mathdx' 3cc_1lx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/home/rgrossekunst/miniforge3/envs/pathfinder_cu12.9.1/lib/libmathdx.so 3cc_1lx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/home/rgrossekunst/miniforge3/envs/pathfinder_testing_cu13.0.2/lib/libmathdx.so 3cc_2la64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/wrk/miniforge3/envs/pathfinder_testing_cu12.9.1/lib/libmathdx.so 3cc_2la64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=/wrk/miniforge3/envs/pathfinder_testing_cu13.0.2/lib/libmathdx.so 3cc_3wx64_cu12.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=C:\Users\rgrossekunst\AppData\Local\miniforge3\envs\pathfinder_testing_cu12.9.1\Library\bin\mathdx64_0.dll 3cc_3wx64_cu13.txt:INFO test_load_nvidia_dynamic_lib[mathdx]: abs_path=C:\Users\rgrossekunst\AppData\Local\miniforge3\envs\pathfinder_testing_cu13.0.2\Library\bin\mathdx64_0.dll Unexplained: mathdx for cu13 is available under conda, but not site-packages |
Sorry, something went wrong.
|
@leofang Could you please review the new #1194 (comment)? Unless you find an issue there, I believe this PR is ready for merging. Regarding the "conda && windows && cccl" anomaly: Possibly the implementation could be streamlined, by reworking the anchor_point logic, but what we have here is a strict improvement (because the logic was completely missing before), and polishing the implementation will only be a very minor detail; this surely isn't structural on the pathfinder side. |
Sorry, something went wrong.
|
FWIW (it doesn't really matter for this PR), I looked into this for a few minutes: Unexplained: mathdx for cu13 is available under conda, but not site-packages Bottom line: That's just the way it is right now. |
Sorry, something went wrong.
|
/ok to test |
Sorry, something went wrong.
|
Regarding commit 43e81b4, I logged my thoughts here: |
Sorry, something went wrong.
Sorry, something went wrong.
|
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Closes #1144, #1116
Bump cuda-pathfinder version to 1.3.2
Main changes:
Piggy-backed changes: