FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Optimize exact-capacity safe UTF-16 conversion by perfloop-agent · Pull Request #1019 · simdutf/simdutf · GitHub

Optimize exact-capacity safe UTF-16 conversion - #1019

Open
perfloop-agent wants to merge 2 commits into
simdutf:masterfrom
perfloop:perfloop-pr-open-1wn057c9pa
Open

Optimize exact-capacity safe UTF-16 conversion#1019
perfloop-agent wants to merge 2 commits into
simdutf:masterfrom
perfloop:perfloop-pr-open-1wn057c9pa

Conversation

perfloop-agent commented Aug 4, 2026
edited
Loading

Copy link
Copy Markdown

Summary

convert_utf16_to_utf8_safe currently routes every input through a conservative slicing loop that repeatedly bounds how many code units may be converted into the remaining output space. For the common case — ASCII data with an exactly-sized output buffer — that caution is pure overhead.

This PR adds a fast path ahead of the slicing loop, taken only when all of the following hold:

  • the build has SIMDUTF_FEATURE_ASCII;
  • the output capacity exactly matches the input length (the ASCII 1:1 case);
  • the input is 32–4096 code units;
  • three cheap sampled probes suggest ASCII, confirmed by a full Haswell ASCII validation before the Haswell converter runs.

Inputs that fail any guard fall through to the existing loop unchanged, so non-ASCII, undersized-buffer, and out-of-range inputs keep their current behavior exactly.

Performance

On workload Haswell ASCII UTF-16-to-UTF-8 safe conversion, 256 code units, exact output capacity, median ns/op changed from 139.9 to 42.5; paired median delta 97.5 (at least 19/20 confidence interval 95.8 to 99.0 from 10 pairs).

An independent verification rebuilt both trees and reproduced the shape with seven paired runs: 35–37 ns/op for this change versus 97–100 ns/op for the comparison.

Testing

  • The fast test suite passes (ctest, Release).
  • The safe-conversion tests pass under AddressSanitizer (leak detection on) and UndefinedBehaviorSanitizer.
  • The guards were exercised at their boundaries: capacity-mismatch, sub-32 and over-4096 sizes, and non-ASCII content all take the pre-existing path.

Authored and verified by Perfloop: every claim above was co-measured on both trees and independently re-verified before submission — the full record is public: case_tksym8s1mb. Replies from this account are human-approved, and a human operator is accountable for this contribution.

lemire commented Aug 14, 2026

Copy link
Copy Markdown
Member

@perfloop-agent This is pretty good. I think it needs work, but it is good.

tsenart commented Aug 14, 2026

Copy link
Copy Markdown

@lemire Cool! The agent can deal with feedback if you leave it here.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants


Back | FazBrowse Home | New Git URL