Repository navigation
Rework locale encoding C functions - #158680
vstinner merged 16 commits into
Conversation
Sorry, something went wrong.
Add output_length to _Py_EncodeLocaleEx(), _Py_EncodeUTF8Ex(), encode_current_locale() and encode_ascii(). So unicode_encode_locale() and wstr_to_utf8() can use the output_length, instead of having to compute strlen(). * Add encode_current_locale_impl() to simplify encode_current_locale(). * _Py_EncodeLocaleEx() now sets error_pos and reason if it fails with -1 or -3. * Add tests on Py_EncodeLocale() and Py_DecodeLocale() functions in test_codecs. * Remove reason parameter of _Py_EncodeUTF8Ex(), encode_current_locale() and encode_ascii().
Also revert an useless change
_Py_EncodeUTF8Ex() now uses size_t instead of Py_ssize_t to iterate on the input string. On error, encode_current_locale_impl() now returns ENCODE_ERROR (new constant) instead of DECODE_ERROR. Add comments on the 3 encode functions.
Add more comments.
Check error pos in decode tests.
* Rename _Py_EncodeLocaleEx() to _Py_EncodeLocale() * Rename _Py_DecodeLocaleEx() to _Py_DecodeLocale() * Rename _Py_DecodeUTF8Ex() to _Py_DecodeUTF8() * Rename _Py_EncodeUTF8Ex() to _Py_EncodeUTF8()
Documentation build overview1 file changed ± c-api/sys.html |
Sorry, something went wrong.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 2102314 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F158680%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
Sorry, something went wrong.
|
Good, the test suite passed on almost all CIs. And the few failures are unrelated to this change.
All these Refleak failures have the same root cause: test_capi leaks. In fact, it's a recent regression on the main branch. I wrote #158806 to fix the leak.
Unrelated test_external_inspection issue: see #158574. |
Sorry, something went wrong.
|
Oh, an assertion failed on AMD64 FreeBSD16 3.x: https://buildbot.python.org/#/builders/1857/builds/874 test_blocking (test.test_concurrent_futures.test_interpreter_pool.InterpreterPoolExecutorTest.test_blocking) ... Assertion failed: (count == argsize), function decode_current_locale, file Python/fileutils.c, line 542. Fatal Python error: Aborted Current thread 0x00001cbd9c848810 [InterpreterPoolExec] (most recent call first): File "/home/buildbot/buildarea/3.x.opsec-fbsd16/build/Lib/genericpath.py", line 40 in isfile File "/home/buildbot/buildarea/3.x.opsec-fbsd16/build/Lib/site.py", line 983 in <genexpr> File "/home/buildbot/buildarea/3.x.opsec-fbsd16/build/Lib/site.py", line 977 in _venv File "/home/buildbot/buildarea/3.x.opsec-fbsd16/build/Lib/site.py", line 1108 in main File "/home/buildbot/buildarea/3.x.opsec-fbsd16/build/Lib/site.py", line 1132 in <module> File "<frozen importlib._bootstrap>", line 543 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 754 in exec_module File "<frozen importlib._bootstrap>", line 909 in _load_unlocked File "<frozen importlib._bootstrap>", line 1303 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1344 in _find_and_load Current thread's C stack trace (most recent call first): <cannot get C stack on this system> Extension modules: _testcapi (total: 1) |
Sorry, something went wrong.
|
I created issue #158893 for the assertion failure. |
Sorry, something went wrong.
Add output_length to _Py_EncodeLocaleEx(), _Py_EncodeUTF8Ex(), encode_current_locale() and encode_ascii(). So
unicode_encode_locale() and wstr_to_utf8() can use the output_length, instead of having to compute strlen().