zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) - #64339
zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry)#64339nodejs-github-bot merged 2 commits into
Conversation
Sorry, something went wrong.
Codecov Report❌ Patch coverage is 98.24856% with 73 lines in your changes missing coverage. Please review. @@ Coverage Diff @@
## main #64339 +/- ##
==========================================
+ Coverage 90.17% 90.31% +0.13%
==========================================
Files 746 760 +14
Lines 242776 246928 +4152
Branches 45741 46578 +837
==========================================
+ Hits 218929 223003 +4074
- Misses 15347 15411 +64
- Partials 8500 8514 +14
... and 40 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
This comment was marked as outdated.
Thanks @bakkot !!! I think the time has come for it on the one hand, and on the other I added some „motivation“ links earlier. Here some more detail: Based on this, we can modify the loader to directly load from an archive. If we do that, we get application bundles. pipobscure#5 & pipobscure#6 Which can then in turn be used to easily create application bundles: https://github.com/pipobscure/experimental-sea So the world had changed enough that it‘s worth proposing again. |
Sorry, something went wrong.
|
I like this a lot. It's likely better to split this into 2 PRs, one for Zip support and one for VFS-Zip, so the Zip support could theoretically be backportable on its own. |
Sorry, something went wrong.
This comment was marked as outdated.
|
As per @mcollina I split this into two PRs. I have the vfs-provider ready to go as follow up one (as it depends on this being merged) I also made sure that the streaming side of things was actually as clean as I intended, and added a few more tests. |
Sorry, something went wrong.
|
Might be worth borrowing some tests from other projects: e.g. python, go, info-zip. There's a lot of edge cases with zip files. (Python's tests are larger than this whole PR put together.) Not all directly applicable because this doesn't include the ability to extract to disk, fortunately. |
Sorry, something went wrong.
I've added a bunch more tests and compared to what go/python/info-zip do. (info-zip is cli exercise, so it's not entirely clear what gets tested). I also gave zip.js an once over and concluded that it was too large a file (it originated from separate files that I've had for ages combined into one). So I split it back out so it would be easier to review. And gave that another look. |
Sorry, something went wrong.
Sorry, something went wrong.
Commit Queue failed- Loading data for nodejs/node/pull/64339 ✔ Done loading data for nodejs/node/pull/64339 ----------------------------------- PR info ------------------------------------ Title zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) (#64339) ⚠ Could not retrieve the email or name of the PR author's from user's GitHub profile! Branch pipobscure:ziparchives -> nodejs:main Labels zlib, semver-minor, lib / src, experimental, needs-ci, commit-queue-rebase Commits 2 - zlib: add ZipEntry, ZipFile, and ZipBuffer - zlib: do not cap ZIP streaming reads by the content-size ceiling Committers 1 - Philipp Dunkel <pip@pipobscure.com> PR-URL: https://github.com/nodejs/node/pull/64339 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh> ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/64339 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh> -------------------------------------------------------------------------------- ℹ This PR was created on Tue, 07 Jul 2026 14:54:34 GMT ✔ Approvals: 4 ✔ - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/64339#pullrequestreview-4832884784 ✔ - James M Snell (@jasnell) (TSC): https://github.com/nodejs/node/pull/64339#pullrequestreview-4816314636 ✔ - Marco Ippolito (@marco-ippolito) (TSC): https://github.com/nodejs/node/pull/64339#pullrequestreview-4816360046 ✔ - Aviv Keller (@avivkeller): https://github.com/nodejs/node/pull/64339#pullrequestreview-4833086662 ✔ Last GitHub CI successful ℹ Last Full PR CI on 2026-07-31T23:54:15Z: https://ci.nodejs.org/job/node-test-pull-request/75350/ - Querying data for job/node-test-pull-request/75350/ ✔ Build data downloaded ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ No git cherry-pick in progress ✔ No git am in progress ✔ No git rebase in progress -------------------------------------------------------------------------------- - Bringing origin/main up to date... From https://github.com/nodejs/node * branch main -> FETCH_HEAD ✔ origin/main is now up-to-date - Downloading patch for 64339 From https://github.com/nodejs/node * branch refs/pull/64339/merge -> FETCH_HEAD ✔ Fetched commits as 67af6d1d3917..fb49acaa84c0 -------------------------------------------------------------------------------- [main ca13b3f1fe] zlib: add ZipEntry, ZipFile, and ZipBuffer Author: Philipp Dunkel <pip@pipobscure.com> Date: Wed Jul 8 17:42:06 2026 +0200 36 files changed, 9665 insertions(+) create mode 100644 lib/internal/zip.js create mode 100644 lib/internal/zip/archive.js create mode 100644 lib/internal/zip/binary.js create mode 100644 lib/internal/zip/buffer.js create mode 100644 lib/internal/zip/compression.js create mode 100644 lib/internal/zip/constants.js create mode 100644 lib/internal/zip/content-size.js create mode 100644 lib/internal/zip/dos.js create mode 100644 lib/internal/zip/entry.js create mode 100644 lib/internal/zip/extra-fields.js create mode 100644 lib/internal/zip/file.js create mode 100644 lib/internal/zip/fs-util.js create mode 100644 lib/internal/zip/header-builders.js create mode 100644 lib/internal/zip/headers.js create mode 100644 test/parallel/test-zlib-zip-coverage.js create mode 100644 test/parallel/test-zlib-zip-edgecases.js create mode 100644 test/parallel/test-zlib-zip-encoding.js create mode 100644 test/parallel/test-zlib-zip-experimental-warning.js create mode 100644 test/parallel/test-zlib-zip-files.js create mode 100644 test/parallel/test-zlib-zip-fuzz.js create mode 100644 test/parallel/test-zlib-zip-hardening.js create mode 100644 test/parallel/test-zlib-zip-internals.js create mode 100644 test/parallel/test-zlib-zip-interop.js create mode 100644 test/parallel/test-zlib-zip-metadata.js create mode 100644 test/parallel/test-zlib-zip-property.js create mode 100644 test/parallel/test-zlib-zip-security.js create mode 100644 test/parallel/test-zlib-zip-sync.js create mode 100644 test/parallel/test-zlib-zip-writable.js create mode 100644 test/parallel/test-zlib-zip-zip64.js create mode 100644 test/parallel/test-zlib-zip.js create mode 100644 test/pummel/test-zlib-zip-slow.js [main 5f7867586e] zlib: do not cap ZIP streaming reads by the content-size ceiling Author: Philipp Dunkel <pip@pipobscure.com> Date: Thu Jul 30 21:49:20 2026 +0200 3 files changed, 31 insertions(+), 17 deletions(-) ✔ Patches applied There are 2 commits in the PR. Attempting autorebase. (node:1608) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated. (Use `node --trace-deprecation ...` to show where the warning was created) Rebasing (2/4) Executing: git node land --amend --yes --------------------------------- New Message ---------------------------------- zlib: add ZipEntry, ZipFile, and ZipBuffer
createZipArchive() serializes a sequence of entries into an archive Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
|
Sorry, something went wrong.
Add ZIP archive support to the node:zlib module through three classes and a set of helpers: - ZipEntry: a single archive member, with buffered reads (content()), bounded-memory streaming reads (contentIterator()), and create()/createStream() for building members. - ZipFile: random access to an archive backed by a file descriptor, reading members lazily without retaining their content and writing new members in place; opened with open()/openSync(). - ZipBuffer: a zero-copy, in-memory view over an archive already held in a Buffer. createZipArchive() serializes a sequence of entries into an archive byte stream, and setMaxZipContentSize() bounds the default in-memory decompression size. Every operation has both an asynchronous and a synchronous form. Signed-off-by: Philipp Dunkel <pip@pipobscure.com> PR-URL: #64339 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh>
Add ZIP archive support to the node:zlib module through three classes and a set of helpers: - ZipEntry: a single archive member, with buffered reads (content()), bounded-memory streaming reads (contentIterator()), and create()/createStream() for building members. - ZipFile: random access to an archive backed by a file descriptor, reading members lazily without retaining their content and writing new members in place; opened with open()/openSync(). - ZipBuffer: a zero-copy, in-memory view over an archive already held in a Buffer. createZipArchive() serializes a sequence of entries into an archive byte stream, and setMaxZipContentSize() bounds the default in-memory decompression size. Every operation has both an asynchronous and a synchronous form. Signed-off-by: Philipp Dunkel <pip@pipobscure.com> PR-URL: #64339 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh>
Add ZIP archive support to the node:zlib module through three classes and a set of helpers: - ZipEntry: a single archive member, with buffered reads (content()), bounded-memory streaming reads (contentIterator()), and create()/createStream() for building members. - ZipFile: random access to an archive backed by a file descriptor, reading members lazily without retaining their content and writing new members in place; opened with open()/openSync(). - ZipBuffer: a zero-copy, in-memory view over an archive already held in a Buffer. createZipArchive() serializes a sequence of entries into an archive byte stream, and setMaxZipContentSize() bounds the default in-memory decompression size. Every operation has both an asynchronous and a synchronous form. Signed-off-by: Philipp Dunkel <pip@pipobscure.com> PR-URL: #64339 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh>
Notable changes: benchmark: * (SEMVER-MINOR) add --analyze mode to compare.js (James M Snell) #65416 crypto: * update root certificates to NSS 3.126 (Node.js GitHub Bot) #65495 * (SEMVER-MINOR) enable SIV and GCM-SIV modes in Cipher/Decipher APIs (Filip Skokan) #63411 diagnostics_channel: * mark TracingChannel as stable (Abdelrahman Awad) #64525 lib,src: * (SEMVER-MINOR) improve histogram implementation (James M Snell) #65024 net: * (SEMVER-MINOR) improve performance of net.BlockList (James M Snell) #64974 perf_hooks: * (SEMVER-MINOR) add statistical hypothesis testing to histogram (James M Snell) #65416 repl: * add basic syntax highlighting (Aviv Keller) #64591 sqlite: * (SEMVER-MINOR) add StatementSync.prototype.close() (Guilherme Araújo) #64232 * (SEMVER-MINOR) add StatementSync.prototype[Symbol.dispose]() (Guilherme Araújo) #64232 util: * (SEMVER-MINOR) add non-throwing MIMEType.parse (James M Snell) #64965 zlib: * (SEMVER-MINOR) add ZipEntry, ZipFile, and ZipBuffer (Philipp Dunkel) #64339 PR-URL: #65551
Summary
Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:
bounded-memory streaming reads (contentIterator()), and
create()/createStream() for building members.
reading members lazily without retaining their content and writing
new members in place; opened with open()/openSync().
in a Buffer.
createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.
P.S.: my CLA should be on file and I wrote this myself so COO is declared