FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Rust I18N Rewrite by logan-gatlin · Pull Request #7052 · cloudflare/workerd · GitHub

Rust I18N Rewrite - #7052

Draft
logan-gatlin wants to merge 3 commits into
mainfrom
logan/rust-i18n
Draft

Rust I18N Rewrite#7052
logan-gatlin wants to merge 3 commits into
mainfrom
logan/rust-i18n

Conversation

logan-gatlin commented Aug 18, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

No description provided.

Ports workerd::api::node::i18n::transcode to a new Rust crate, //src/rust/i18n, selected at runtime by a new NODEJS_I18N_RUST autogate. The C++ implementation is left in place, byte for byte, as the gate-off/rollback path.

The dispatch table, output-size computation, ICU substitute-character setup, empty-input handling, ISOLATE_LIMIT checks, and truncation all move to Rust (src/rust/i18n/dispatch.rs). The underlying codecs stay the same: a new C++ shim (src/rust/i18n/shim.{h,c++}) exposes the ICU ucnv_* primitives and the four simdutf functions the C++ path already uses, so both paths call identical codecs and cannot silently diverge.

i18n::transcode gets a gate branch at the top, following the createNodeException pattern: it maps api::node::Encoding to the bridge's Encoding enum via a fromImpl overload (kj-rs/convert.h idiom) whose switch has no default arm, rejecting BASE64/BASE64URL/HEX on either side (a deliberate divergence from the unmodified C++ dispatch, which only checks the from encoding). The Rust entry point returns a jsg::v8::ffi::MaybeLocal naming a Uint8Array; ffi-inl.h gains the missing Rust-to-C++ MaybeLocal conversion (maybe_local_from_ffi), mirroring the existing local_from_ffi.

10 Rust unit tests in dispatch.rs cover every (from, to) pair, empty input, unmappable-character substitution, the ASCII->UTF16LE latin1-widening quirk, the UTF8->UTF16LE zero-estimate quirk, invalid UTF-8, odd-length UTF-16LE input, and unpaired surrogates.
Point the src/rust/i18n shim's ICU dependency at :icuuc instead of the umbrella :icu target, per R7 (the shim only uses ucnv_* primitives from ICU's common library, not icui18n).

The bazel/BUILD.icu file vendored via V8's patch set only marks :icu and :icudata public; :icuuc is package-private. Add a small new V8 patch (0039) granting :icuuc public visibility, following the existing precedent of patch 0027 doing the same for :icudata.

Verified: just clippy i18n is clean; bazel test //src/rust/i18n:i18n_test, //src/workerd/api/node/tests:buffer-nodejs-test@, and @all-autogates all pass; full bazel test //... passes 1623/1623, observed running fresh (no cached results).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant


Back | FazBrowse Home | New Git URL