| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
We require contributors to sign our Contributor License Agreement, and we don't have @ondraulehla on file. You can sign our CLA at https://e2b.dev/docs/cla . Once you've signed, post a comment here that says '@cla-bot check' |
Sorry, something went wrong.
readLines() grew a single string with `buffer += chunk` and re-scanned it on every chunk, making large outputs O(n^2): a 22 MB single-line stdout took ~30s and OOM-killed memory-constrained hosts (e2b-dev#251). It also created a fresh TextDecoder per chunk, so multi-byte UTF-8 sequences split across chunk boundaries decoded as U+FFFD. Accumulate decoded fragments in an array (O(1) append, joined only when a line completes), scan for newlines only within the new chunk, and reuse one TextDecoder with { stream: true }, flushing it at stream end. Fixes e2b-dev#251 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
We require contributors to sign our Contributor License Agreement, and we don't have @ondraulehla on file. You can sign our CLA at https://e2b.dev/docs/cla . Once you've signed, post a comment here that says '@cla-bot check' |
Sorry, something went wrong.
|
The cla-bot has been summoned, and re-checked this pull request! |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Fixes #251
Problem
readLines() in js/src/utils.ts accumulates the response body with buffer += chunk and re-scans the whole buffer on every chunk. String append plus full re-scan is O(n^2): a large single-line stdout (the exact shape produced by print("x" * 22_000_000)) causes multi-second event-loop stalls and OOM kills on memory-constrained hosts, as reported in #251 (Linear: #251).
There is a second, related bug: a fresh new TextDecoder() is created for every chunk, so a multi-byte UTF-8 sequence split across a chunk boundary (any emoji, most non-Latin scripts) decodes as U+FFFD replacement characters.
Fix
Semantics are unchanged: the same lines are yielded for embedded, trailing, and consecutive newlines, and for the final unterminated remainder.
Benchmark
Node v24, 64 KiB chunks, single-line payload terminated by \n (the #251 scenario):
The old implementation scales roughly 4x per size doubling (quadratic); the new one scales linearly and is 77x faster at 88 MB.
Tests
Added js/tests/utils.test.ts with 9 unit tests covering line splitting across chunk boundaries, empty lines, trailing text without a newline, empty streams, multi-byte UTF-8 split across chunks, incomplete sequences at stream end, and a ~8 MB many-chunk case. These are pure unit tests over a synthetic ReadableStream, so no sandbox or API key is needed. pnpm test tests/utils.test.ts, pnpm build, and pnpm lint all pass.