Repository navigation
Avoid removed Py_FileSystemDefaultEncoding in load_library - #287
Merged
mattip merged 2 commits intoOct 24, 2026
Merged
mattip merged 2 commits into
mattip merged 2 commits into
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 18, 2026
This was referenced Oct 18, 2026
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 18, 2026
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
9159b58 to
c12ae81
Compare
October 18, 2026 18:47
ozeranskii
added a commit
to ozeranskii/cffi
that referenced
this pull request
Oct 19, 2026
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
c12ae81 to
46d7a1e
Compare
October 19, 2026 17:59
CPython 3.16 removed Py_FileSystemDefaultEncoding from the headers, so _cffi_backend.c no longer compiled. Convert the file name with PyUnicode_FSConverter instead, which uses the filesystem encoding, rejects embedded null characters and is available on all supported versions. The encoded name is kept in *p_temp, which the callers already release.
ozeranskii
force-pushed
the
fix-py316-fsdefaultencoding
branch
from
46d7a1e to
38dd2f6
Compare
October 19, 2026 18:36
Copy link
Copy Markdown
Contributor
Author
|
@mattip, could you approve the CI workflows for the latest push? The first run failed only in clang_TSAN, on a non-ASCII dlopen test I've since removed: it loaded and closed a copy of the extension from several threads at once, and TSan flagged that inside the dynamic loader. Non-ASCII file names are already covered by test_re_python.py and test_ownlib.py. All other jobs passed, including Windows and iOS. |
Sorry, something went wrong.
Copy link
Copy Markdown
|
On Fedora Linux 46 x86_64, cffi builds successfully with this patch and Python 3.16.0a1 (1975 passed, 103 skipped). |
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CPython 3.16 removed Py_FileSystemDefaultEncoding from the headers (python/cpython#158020), so _cffi_backend.c no longer compiles there:
This replaces the "et" conversion in b_do_dlopen with "O&" + PyUnicode_FSConverter. That converter encodes with the interpreter's filesystem encoding, rejects embedded null characters, releases its result if a later argument fails to parse, and has the same behaviour on every supported version (3.10+), so no version checks are needed. The None, cdata and Windows wide-char paths are unchanged.
PyUnicode_EncodeFSDefault, as suggested in #86, would be shorter. It does not check for embedded nulls, though, so load_library("/usr/lib/libc.so.6\0x") would silently load the truncated path. Today that raises TypeError; with this change it raises ValueError: embedded null byte, like open(). The new test covers this and the existing rejection of bytes. Non-ASCII file names are already covered by testing/cffi1/test_re_python.py and testing/cffi0/test_ownlib.py.
Addresses the Py_FileSystemDefaultEncoding part of #180 (_PyLong_Sign remains). Refs #86.
Tested on macOS arm64 with python setup.py build_ext -i && pytest src/c/test_c.py on 3.10 through 3.15t (unchanged: 225 passed, 6 skipped) and on 3.16.0a1 / 3.16.0a1t (previously failed to build, now 225 passed, 6 skipped). I have not tested on Windows, where a str goes through the separate dlopenWinW branch, which this does not touch. That branch already truncates at an embedded null, so the test skips that check on win32. This PR does not add 3.16 to CI, since neither cibuildwheel nor setup-python ship 3.16 yet. I'm happy to send that as a follow-up once they do.