Repository navigation
[MNT]: Pybind11 transition plan #23846
Description
We intend to switch the C/C++ extension build system to use pybind11 as discussed in this week's call. There is a demonstration in PR #23787 showing that the approach works and listing the pros and cons. It is too much work to do in a single PR so will probably be done one extension at a time. This issue is intended to track the overall progress by editing this post to tick the relevant check box and add the PR number for each extension completed.
C/C++ extensions:
- agg (start in Convert Agg extension to pybind11 #27011)
- c_internal_utils (PR in Use pybind11 in _c_internal_utils module #26794)
- ft2font (PR in Convert ft2font extension to pybind11 #28785)
- image (implemented in Use pybind11 in image module #26275)
- path Convert path extension to pybind11 #27087
- qhull(PR in Use pybind11 for qhull wrapper #27184)
- tkagg (PR in Convert TkAgg utilities to pybind11 #26992)
- tri (implemented in Use pybind11 for tri module #24522)
- ttconv (implemented in Use pybind11 in ttconv module #25253)
- macosx (developer needs access to macOS)
Other work required:
- Update to use PEP517/518-complicant build system. Not strictly required but makes the build process simpler so ideally would be done before merging any specific extension PRs. Work on this has started in Start transitioning to pyproject.toml #23829; for pybind11 we only really need the [build-system] section of pyproject.toml which was added in Simplest pyproject.toml containing build-system only #24102.
- Remove unneeded wrapper code in src directory such as the array_view class when all extensions completed. Pybind11 cleanup #28857
- Remove numpy as a build-time dependency when all extensions completed as pybind11 does not require it. Pybind11 cleanup #28857
Activity
QuLogic commented on Mar 30, 2023
On a related note, has anyone tried out nanobind? It's probably a bit too new for us, but curious if anyone had any opinions on it.
matthewfeickert commented on Mar 30, 2023
I just today migrated a (small) research project to nanobin v1.0.0.
As this project is array heavy nanobin is super nice for this as it allows you to just use the nb::ndarray API and avoid all the buffer stuff in pybind11 (which honestly isn't that bad, but if you don't have to do it yourself that's great).
Other advantages that are nice about nanobind (YMMV though project to project):
My first impressions are that it is great!
QuLogic commented on Oct 5, 2023
I'm not sure about this one, as it doesn't use NumPy, and is written in Objective C, not C or C++.
greglucas commented on Oct 6, 2023
I agree pybind doesn't seem like the right fit. I have wondered if it'd be worthwhile to transition to use pyobjc at all so we could write the extension in Python code... https://pyobjc.readthedocs.io/en/latest/
Of course, that would mean an extra dependency on macos and needing to learn how to write that style of Python.
QuLogic commented on Sep 6, 2024
The last PR #28785 does ft2font. After that there will be some clean up to remove the NumPy wrappers.
QuLogic commented on Sep 24, 2024
The last two PRs: #28856 takes care of porting remaining common functionality to pybind11, and #28857 removes NumPy and the helpers.