| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
Every CJK decoder emits output through OUTCHAR(), which called the exported _PyUnicodeWriter_WriteChar(). The codecs are shared extensions, so that is a cross-module call for each output character, and the function is nothing but the _PyUnicodeWriter_Prepare() macro plus a store that the caller could have done itself. OUTCHAR2, directly below it, was already written inline for exactly this reason. Use _PyUnicodeWriter_WriteCharInline() instead. One macro, and it applies to every codec in the package. | benchmark | before | after | change | spread | |-------------------------|----------:|----------:|-------:|-------:| | big5 decode 1.8MB | 6.32 ms | 2.56 ms | -59% | 0.3pp | | euc_kr decode 1.8MB | 6.94 ms | 3.18 ms | -54% | 0.1pp | | shift_jis decode 1.8MB | 6.93 ms | 3.17 ms | -54% | 0.4pp | | euc_jp decode 1.8MB | 6.32 ms | 3.18 ms | -50% | 0.2pp | | gb2312 decode 1.8MB | 6.32 ms | 3.18 ms | -50% | 0.3pp | | euc_jp decode kana | 3.26 ms | 2.05 ms | -37% | 4.0pp | | iso2022_jp decode | 9.48 ms | 7.09 ms | -25% | 8.3pp | Through the StreamReader wrappers, where Python-level call overhead dilutes it: | benchmark | before | after | change | |----------------------------|----------:|----------:|-------:| | euc_jp read() [1 call] | 6.33 ms | 3.18 ms | -50% | | euc_jp read(1024) xN | 7.06 ms | 3.93 ms | -44% | | euc_jp readlines() | 8.76 ms | 5.60 ms | -36% | | shift_jis readline() x20k | 11.91 ms | 8.25 ms | -30% | | euc_jp readline() x20k | 11.35 ms | 9.27 ms | -18% | Decoding every 1- and 2-byte sequence (65536 + 256 inputs) under each of the 23 CJK codecs, plus 3-byte probes, hashes identically before and after. Measured with the two builds in separate worktrees. These codecs live in shared .so files, so copying the python executable within one build tree does not isolate them: both copies load whichever module was built last, and the change measures as an exact no-op.
There was a problem hiding this comment.
lgtm
Thank you for doing this.
Sorry, something went wrong.
…write in CJK decoders (pythongh-156538)
| Back | FazBrowse Home | New Git URL |
Every CJK decoder emits output through OUTCHAR(), which called the exported _PyUnicodeWriter_WriteChar(). Since the codecs are shared extension modules, that's a cross boundary call to the exported function for each output character. Swapping to the _PyUnicodeWriter_WriteCharInline() removes the overhead of crossing the boundary speeding up decoding.
The single macro applies to all of the codec modules.