FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Implement TS DigestStream by jasnell · Pull Request #7119 · cloudflare/workerd · GitHub

Implement TS DigestStream - #7119

Merged
jasnell merged 3 commits into
mainfrom
jasnell/ts-streams-digeststream
Aug 25, 2026
Merged

Implement TS DigestStream#7119
jasnell merged 3 commits into
mainfrom
jasnell/ts-streams-digeststream

Conversation

jasnell commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Implement the TypeScript-version of DigestStream

Also includes two fixes to both implementations, one dealing with the case-insensitive matching for the crc algorithm names, and the second a new option to handle split surrogates.

jasnell requested review from a team as code owners August 25, 2026 17:16
DigestStream is one of only two WritableStream subclasses in the runtime.
When the typescript_implemented_streams flag swaps globalThis.WritableStream
for the TypeScript class, a C++ subclass of the C++ WritableStream stops
passing the brand checks pipeTo relies on, and instanceof WritableStream
becomes false for it. Reimplementing the subclass in TypeScript is what
restores the hierarchy.

Only the stream semantics move. Hashing stays in C++ behind a new
DigestContextHandle, reached through a deliberately crypto-free declaration
in digest-bootstrap.h so the per-isolate bootstrap need not depend on the
crypto headers. String chunks are forwarded to it rather than encoded here,
because the encoding is not expressible in JavaScript: TextEncoder always
substitutes U+FFFD, so it cannot produce the WTF-8 bytes DigestStream has
always hashed. That is also why update() returns a byte count, a string's
UTF-8 length not being observable from JavaScript.

Both implementations run crypto-streams-test.js unmodified so the two
cannot drift. ts-digest-stream-test.js covers TypeScript-only behavior and
holds the canary that the install in main.ts actually took effect, without
which the rest of the suite would pass against the C++ class too.

One divergence is intentional: the digest promise is marked handled, so
abandoning it produces no unhandled-rejection report where the C++
implementation does. Consuming digest is optional, an abort still surfaces
through the writer's ready/closed promises, and marking does not propagate
to derived promises.

The other subclass, FileSystemWritableFileStream, has the same problem but
is constructed from C++ rather than by user code, so it needs a native
source/sink bridge instead of a straight port.
String chunks are hashed as WTF-8, which encodes a lone surrogate
literally (U+D800 becomes ED A0 80) rather than substituting U+FFFD as
TextEncoder and every other UTF-8 producer would. That stays the default,
since changing it would silently alter the digests of existing callers,
but `new DigestStream(algorithm, {toWellFormed: true})` now opts into the
substitution.

Under toWellFormed the conversion is a streaming text encoder, so a
surrogate pair split across two writes is joined rather than being
mistaken for two unpaired surrogates: a lead surrogate ending a chunk is
held back until the next chunk settles whether it pairs, and flushed as
U+FFFD at end of stream if it never does. This mirrors TextEncoderStream.
The default encoding stays stateless and joins nothing, which is why the
two can disagree on bytesWritten across chunks -- 4 bytes for a joined
pair against 3+3 for two lone surrogates.

Both DigestStream implementations share DigestStringEncoder, so they
cannot drift apart under either encoding. The shared test file pins the
default (stringChunksAreNotTextEncodedByDefault) and the opt-in
(toWellFormedMatchesTextEncoder) against both, along with JSG's option
bag rules, which the TypeScript side has to reproduce by hand.
The CRC names were compared exactly while every other name went to
EVP_get_digestbyname, which is case-insensitive. So md5 and MD5 both
worked but CRC32 did not, an inconsistency with nothing behind it.

No compatibility flag is needed because the change only widens what is
accepted: every name that worked before still selects the same algorithm,
and the newly accepted spellings previously threw. The pre-existing casing
test confirmed CRC32 threw, which rules out an OpenSSL digest already
answering to that name and now being shadowed by a CRC.

Both implementations reach newDigestContext, so this fixes them together.
The tests compare digests rather than just checking that the constructor
did not throw, since a casing that silently selected a different algorithm
would otherwise pass.
jasnell force-pushed the jasnell/ts-streams-digeststream branch from 7ec1088 to d414e8f Compare August 25, 2026 17:20

This comment was marked as low quality.

jasnell requested a review from npaun August 25, 2026 19:01
jasnell merged commit 917a89c into main Aug 25, 2026
68 of 71 checks passed
jasnell deleted the jasnell/ts-streams-digeststream branch August 25, 2026 22:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants


Back | FazBrowse Home | New Git URL