module: implement NODE_COMPILE_CACHE for automatic on-disk code caching by joyeecheung · Pull Request #52535 · nodejs/node · GitHub

This is the initial PR for implementing the idea proposed in #47472. It takes inspiration from ccache, the v8-compile-cache npm package (but faster, supports ESM and replaces CJS loader moneky-patching in user land) and v8 code caching in Blink. The first commit is backported from https://chromium-review.googlesource.com/c/v8/v8/+/5401780 which has already landed in the upstream (this is required for import() to work in deserialized code). For more motivation and background of this PR, check out #47472

Local numbers:

❯ hyperfine "./out/Release/node test/fixtures/snapshot/typescript.js" --warmup 5
Benchmark 1: ./out/Release/node test/fixtures/snapshot/typescript.js
  Time (mean ± σ):     124.3 ms ±  13.2 ms    [User: 109.6 ms, System: 11.6 ms]
  Range (min … max):   116.1 ms … 170.2 ms    21 runs

❯ export NODE_COMPILE_CACHE=/tmp/.node_compile_cache
❯ hyperfine "./out/Release/node test/fixtures/snapshot/typescript.js" --warmup 5
Benchmark 1: ./out/Release/node test/fixtures/snapshot/typescript.js
  Time (mean ± σ):      72.3 ms ±   3.2 ms    [User: 62.3 ms, System: 10.2 ms]
  Range (min … max):    70.5 ms …  87.2 ms    40 runs

There are more numbers from folks trying on an earlier iteration of this work (although this branch now hashes the code and the cache as an extra check, but crc32 should be fast enough that it's not too big of an overhead). For example from #47472 (comment) tsc --version can go from ~90ms to ~40ms, yarn --version can go from ~190ms to ~135ms, from #47472 (comment) npm run echo with a ncc-bundled npm may go from ~150ms to ~110ms

summary

This patch implements automatic on-disk code caching that can be enabled
via an environment variable NODE_COMPILE_CACHE.

When set, whenever Node.js compiles a CommonJS or a ECMAScript Module,
it will use on-disk V8 code cache persisted in the specified directory
to speed up the compilation. This may slow down the first load of a
module graph, but subsequent loads of the same module graph may get
a significant speedup if the contents of the modules do not change.
Locally, this speeds up loading of test/fixtures/snapshot/typescript.js
from ~130ms to ~80ms.

To clean up the generated code cache, simply remove the directory.
It will be recreated the next time the same directory is used for
NODE_COMPILE_CACHE.

Compilation cache generated by one version of Node.js may not be used
by a different version of Node.js. Cache generated by different versions
of Node.js will be stored separately if the same directory is used
to persist the cache, so they can co-exist.

Caveat: currently when using this with V8 JavaScript code coverage, the
coverage being collected by V8 may be less precise in functions that are
deserialized from the code cache. It's recommended to turn this off when
running tests to generate precise coverage.

Implementation details:

There is one cache file per module on disk. The directory layout
is:

Compile cache directory (from NODE_COMPILE_CACHE)
- 8b23c8fe: CRC32 hash of CachedDataVersionTag + NODE_VERESION
- 2ea3424d:
  - 10860e5a: CRC32 hash of filename + module type
  - 431e9adc: ...
  - ...

Inside the cache file, there is a header followed by the actual
cache content:

[uint32_t] code size
[uint32_t] code hash
[uint32_t] cache size
[uint32_t] cache hash
... compile cache content ...

When reading the cache file, we'll also check if the code size
and code hash match the code that the module loader is loading
and whether the cache size and cache hash match the file content
read. If they don't match, or if V8 rejects the cache passed,
we'll ignore the mismatch cache, and regenerate the cache after
compilation succeeds and rewrite it to disk.

There are still a bunch of TODOs for this feature, some of them are documented as TODO comments. In addition we can look into idle-time code cache serialization/writing which is also done by Blink to reduce the impact on first load. But this should be good enough as an initial iteration.

Note: why CRC32? Because we already have it in the dependencies (in zlib.h), it's fast and can avoid enough collisions for this use case. We do have some existing hashing functions available via openssl but those won't be available on no-crypto builds. We can revisit if CRC32 turns out to be inadequate for this feature. We can also revisit the directory structure or use an actual db, but for now the file I/O cost doesn't really show up when I profile it locally. The remaining bottleneck is the repeated UTF8 encoding, which I plan to get rid of in a follow-up (because it needs to dance with monkey-patching).

Refs: #47472

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7

This patch implements automatic on-disk code caching that can be enabled via an environment variable NODE_COMPILE_CACHE. When set, whenever Node.js compiles a CommonJS or a ECMAScript Module, it will use on-disk [V8 code cache][] persisted in the specified directory to speed up the compilation. This may slow down the first load of a module graph, but subsequent loads of the same module graph may get a significant speedup if the contents of the modules do not change. Locally, this speeds up loading of test/fixtures/snapshot/typescript.js from ~130ms to ~80ms. To clean up the generated code cache, simply remove the directory. It will be recreated the next time the same directory is used for `NODE_COMPILE_CACHE`. Compilation cache generated by one version of Node.js may not be used by a different version of Node.js. Cache generated by different versions of Node.js will be stored separately if the same directory is used to persist the cache, so they can co-exist. Caveat: currently when using this with V8 JavaScript code coverage, the coverage being collected by V8 may be less precise in functions that are deserialized from the code cache. It's recommended to turn this off when running tests to generate precise coverage. Implementation details: There is one cache file per module on disk. The directory layout is: - Compile cache directory (from NODE_COMPILE_CACHE) - 8b23c8fe: CRC32 hash of CachedDataVersionTag + NODE_VERESION - 2ea3424d: - 10860e5a: CRC32 hash of filename + module type - 431e9adc: ... - ... Inside the cache file, there is a header followed by the actual cache content: ``` [uint32_t] code size [uint32_t] code hash [uint32_t] cache size [uint32_t] cache hash ... compile cache content ... ``` When reading the cache file, we'll also check if the code size and code hash match the code that the module loader is loading and whether the cache size and cache hash match the file content read. If they don't match, or if V8 rejects the cache passed, we'll ignore the mismatch cache, and regenerate the cache after compilation succeeds and rewrite it to disk.

Review requested:

@nodejs/gyp
@nodejs/loaders
@nodejs/security-wg
@nodejs/v8-update
@nodejs/vm

CI: https://ci.nodejs.org/job/node-test-pull-request/58387/

When enabling NODE_COMPILE_CACHE and running tools/test.py parallel locally, three tests would fail:

Failed tests:
out/Release/node /Users/joyee/projects/node/test/parallel/test-runner-coverage.js
out/Release/node /Users/joyee/projects/node/test/parallel/test-runner-output.mjs
out/Release/node /Users/joyee/projects/node/test/parallel/test-v8-coverage.js

The failed tests are all related to less precise coverage which is currently a known caveat of code cache + v8 native coverage collection (e.g. see #51672). So I just documented that a bit in the docs that users should not use the two together for now. I could spare some time later to fix it in V8, but I don't think it's a blocker for this PR because this feature is currently opt-in. User-land code caching can run into this problem too, and it's technically a V8 bug.

There are more numbers from folks trying on an earlier iteration of this work

Some up-to-date results with f7806b3:

$ hyperfine -w 10 "NODE_COMPILE_CACHE='' ./node ./.yarn/releases/yarn-4.1.1.cjs --version" "NODE_COMPILE_CACHE='/tmp/node-cache' ./node ./.yarn/releases/yarn-4.1.1.cjs --version"
Benchmark 1: NODE_COMPILE_CACHE='' ./node ./.yarn/releases/yarn-4.1.1.cjs --version
  Time (mean ± σ):     165.5 ms ±   1.9 ms    [User: 156.5 ms, System: 25.4 ms]
  Range (min … max):   162.2 ms … 169.3 ms    18 runs

Benchmark 2: NODE_COMPILE_CACHE='/tmp/node-cache' ./node ./.yarn/releases/yarn-4.1.1.cjs --version
  Time (mean ± σ):     116.0 ms ±   1.3 ms    [User: 109.2 ms, System: 23.5 ms]
  Range (min … max):   113.8 ms … 119.7 ms    25 runs

Summary
  NODE_COMPILE_CACHE='/tmp/node-cache' ./node ./.yarn/releases/yarn-4.1.1.cjs --version ran
    1.43 ± 0.02 times faster than NODE_COMPILE_CACHE='' ./node ./.yarn/releases/yarn-4.1.1.cjs --version

This is so rad! Having had to manage and hook into systems for this having it built-in is very nice! ❤️ it.

This description is very technical - I would use more layman terms for the first paragraph.

Do you have specific suggestions? I don't want to make it sound too magical to avoid false advertisement.

i'll do a follow up PR after this lands to not hold it but ideally something that focuses more on what it does and not the technical implementation

LGTM as experimental/active development. Have a few questions:

how does this interact with SEA (would it make sense or be possible to only eventually bundle the code cache and not the code?)
What happens on overflows or tampering with the cache file? Is there a security concern here regarding passing different code through the cache file (since the hash is just a crc32)?
How does this feature interact with loaders/require hooks that modify the file contents and in particular passing them in different order? (reading the code it should work but want to make sure)

(Not blocking for this PR - we should have better user facing errors if there are permission issues creating/reading/writing the cache)

This should not error at all, the idea is that if caching cannot be done, this is just a no-op. There can be some logging to help users figure out why the cache isn't hit to ensure better performance, but the cache not being hit only leads to "not speeding up module loading", and it's not an error (just like when you write code that V8 cannot optimize or have to deopt with, there is no error either).

It's literally caching I get that but when it won't work for users and they'll want to figure out why they'll ask us. It would be useful to at least trace it as as. warning or add DEBUG* like logs.

There are already traces. The handful of tests here are testing against the traces .

how does this interact with SEA (would it make sense or be possible to only eventually bundle the code cache and not the code?)

Currently SEA only supports bundling a single script and it's not compiled using the normal CJS loader, but there's already a dedicated option in the SEA config to enable code cache for that one file ("useCodeCache": true). Neither static import nor dynamic import is not currently supported in SEA. Users can load on-disk CJS files via createRequire() and in that case if the environment variable is enabled, the cache of on-disk modules loaded through the created require() will be put in that directory. This is all separated from the SEA itself.

What happens on overflows or tampering with the cache file? Is there a security concern here regarding passing different code through the cache file (since the hash is just a crc32)?

The cache will be silently ignored if the hash doesn't match. Under the Node.js threat model, code loaded from the file system is already trusted. If you can gain access to the file system, you might as well just replace the source file on disk that'll be executed and do whatever you want in that replaced file, instead of wasting all your time figuring out how to mutate the code cache that meets all the requirement below:

Can do what you need (this means you need to know how to hand-write V8 bytecode to do what you need. Also, it must be run with the right V8 version since the bytecode format changes all the time)
The mutated code cache is still of the same length of the original code cache (so if you want to do a lot of additional stuff in that bytecode, this is unlikely to pass)
It passes the additional checks done by V8, and V8 can actually run it just fine (and once the V8 version in Node.js is upgraded this is likely to stop working)
The CRC32 of this new code cache must also conflict with the original code cache (finding CRC32 conflicts isn't that hard, but finding CRC32 conflicts that meet 1-3 is hard)

TBH it seems a bit silly for an attacker to waste all their effort on this when they can already just replace the source code on disk if they have access to the file system. Also, this is already what people have been doing in the wild via e.g. https://www.npmjs.com/package/v8-compile-cache with 12M downloads/week, so if there are any security concerns, well it's already out there for many years, and it's unlikely that the ecosystem will just stop relying on a capability like this.

How does this feature interact with loaders/require hooks that modify the file contents and in particular passing them in different order? (reading the code it should work but want to make sure)

If they are using the "official" hooks, e.g. module.register the modules will be cached and the resource name & module content cached match the results coming out of the loaders. If it's monkey-patching CJS hooks this will be ignored if they patch Module.wrap, Module.wrapper or Module.prototype._compile (without calling the original version, this means it's a no-op if v8-compile-cache is used since that package does this), but otherwise it should work fine.

Would it be feasible to add a way to enable this for the current process only, perhaps as a follow-up?

For context Corepack uses v8-compile-cache to speed up the loading of package managers but doesn't modify process.env to avoid affecting anything said package managers might spawn.

Would it be feasible to add a way to enable this for the current process only, perhaps as a follow-up?

Yeah I think we can discuss other ways to enable this (or a possible solution to enable it by default and pointing it to somewhere that's less likely to fill up the disk e.g. the tmpdir) in a follow-up. For the initial PR an opt-in environment variable should be enough.

…e caching

CI: https://ci.nodejs.org/job/node-test-pull-request/58449/

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @joyeecheung.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

Joyee: See the PR description

Thanks Joyee, appreciate all the input.

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7 PR-URL: #52535 Refs: #47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> PR-URL: #52293 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7 PR-URL: nodejs#52535 Refs: nodejs#47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7 PR-URL: #52535 Refs: #47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> PR-URL: #52293 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7 PR-URL: nodejs#52535 Refs: nodejs#47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>

Original commit message: [compiler] reset script details in functions deserialized from code cache During the serialization of the code cache, V8 would wipe out the host-defined options, so after a script id deserialized from the code cache, the host-defined options need to be reset on the script using what's provided by the embedder when doing the deserializing compilation, otherwise the HostImportModuleDynamically callbacks can't get the data it needs to implement dynamic import(). Change-Id: I33cc6a5e43b6469d3527242e083f7ae6d8ed0c6a Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5401780 Reviewed-by: Leszek Swirski <leszeks@chromium.org> Commit-Queue: Joyee Cheung <joyee@igalia.com> Cr-Commit-Position: refs/heads/main@{#93323} Refs: v8/v8@cd10ad7 PR-URL: #52535 Refs: #47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com> PR-URL: #52465 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>

This patch implements automatic on-disk code caching that can be enabled via an environment variable NODE_COMPILE_CACHE. When set, whenever Node.js compiles a CommonJS or a ECMAScript Module, it will use on-disk [V8 code cache][] persisted in the specified directory to speed up the compilation. This may slow down the first load of a module graph, but subsequent loads of the same module graph may get a significant speedup if the contents of the modules do not change. Locally, this speeds up loading of test/fixtures/snapshot/typescript.js from ~130ms to ~80ms. To clean up the generated code cache, simply remove the directory. It will be recreated the next time the same directory is used for `NODE_COMPILE_CACHE`. Compilation cache generated by one version of Node.js may not be used by a different version of Node.js. Cache generated by different versions of Node.js will be stored separately if the same directory is used to persist the cache, so they can co-exist. Caveat: currently when using this with V8 JavaScript code coverage, the coverage being collected by V8 may be less precise in functions that are deserialized from the code cache. It's recommended to turn this off when running tests to generate precise coverage. Implementation details: There is one cache file per module on disk. The directory layout is: - Compile cache directory (from NODE_COMPILE_CACHE) - 8b23c8fe: CRC32 hash of CachedDataVersionTag + NODE_VERESION - 2ea3424d: - 10860e5a: CRC32 hash of filename + module type - 431e9adc: ... - ... Inside the cache file, there is a header followed by the actual cache content: ``` [uint32_t] code size [uint32_t] code hash [uint32_t] cache size [uint32_t] cache hash ... compile cache content ... ``` When reading the cache file, we'll also check if the code size and code hash match the code that the module loader is loading and whether the cache size and cache hash match the file content read. If they don't match, or if V8 rejects the cache passed, we'll ignore the mismatch cache, and regenerate the cache after compilation succeeds and rewrite it to disk. PR-URL: #52535 Refs: #47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>

Notable changes: buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) #52428 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) #52492 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) #52618 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) #52509 module: * (SEMVER-MINOR) implement NODE_COMPILE_CACHE for automatic on-disk code caching (Joyee Cheung) #52535 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) #52474 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) #52595 src,permission: * throw async errors on async APIs (Rafael Gonzaga) #52730 test_runner: * (SEMVER-MINOR) add --test-skip-pattern cli option (Aviv Keller) #52529 url: * (SEMVER-MINOR) implement parse method for safer URL parsing (Ali Hassan) #52280 PR-URL: TODO

Notable changes: buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) #52428 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) #52492 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) #52618 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) #52509 module: * (SEMVER-MINOR) implement NODE_COMPILE_CACHE for automatic on-disk code caching (Joyee Cheung) #52535 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) #52474 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) #52595 src,permission: * throw async errors on async APIs (Rafael Gonzaga) #52730 test_runner: * (SEMVER-MINOR) add --test-skip-pattern cli option (Aviv Keller) #52529 url: * (SEMVER-MINOR) implement parse method for safer URL parsing (Ali Hassan) #52280 PR-URL: #52768

Notable changes: buffer: * improve `base64` and `base64url` performance (Yagiz Nizipli) nodejs#52428 dns: * (SEMVER-MINOR) add order option and support ipv6first (Paolo Insogna) nodejs#52492 events,doc: * mark CustomEvent as stable (Daeyeon Jeong) nodejs#52618 lib, url: * (SEMVER-MINOR) add a `windows` option to path parsing (Aviv Keller) nodejs#52509 module: * (SEMVER-MINOR) implement NODE_COMPILE_CACHE for automatic on-disk code caching (Joyee Cheung) nodejs#52535 net: * (SEMVER-MINOR) add CLI option for autoSelectFamilyAttemptTimeout (Paolo Insogna) nodejs#52474 src: * (SEMVER-MINOR) add `string_view` overload to snapshot FromBlob (Anna Henningsen) nodejs#52595 src,permission: * throw async errors on async APIs (Rafael Gonzaga) nodejs#52730 test_runner: * (SEMVER-MINOR) add --test-skip-pattern cli option (Aviv Keller) nodejs#52529 url: * (SEMVER-MINOR) implement parse method for safer URL parsing (Ali Hassan) nodejs#52280 PR-URL: nodejs#52768

This patch implements automatic on-disk code caching that can be enabled via an environment variable NODE_COMPILE_CACHE. When set, whenever Node.js compiles a CommonJS or a ECMAScript Module, it will use on-disk [V8 code cache][] persisted in the specified directory to speed up the compilation. This may slow down the first load of a module graph, but subsequent loads of the same module graph may get a significant speedup if the contents of the modules do not change. Locally, this speeds up loading of test/fixtures/snapshot/typescript.js from ~130ms to ~80ms. To clean up the generated code cache, simply remove the directory. It will be recreated the next time the same directory is used for `NODE_COMPILE_CACHE`. Compilation cache generated by one version of Node.js may not be used by a different version of Node.js. Cache generated by different versions of Node.js will be stored separately if the same directory is used to persist the cache, so they can co-exist. Caveat: currently when using this with V8 JavaScript code coverage, the coverage being collected by V8 may be less precise in functions that are deserialized from the code cache. It's recommended to turn this off when running tests to generate precise coverage. Implementation details: There is one cache file per module on disk. The directory layout is: - Compile cache directory (from NODE_COMPILE_CACHE) - 8b23c8fe: CRC32 hash of CachedDataVersionTag + NODE_VERESION - 2ea3424d: - 10860e5a: CRC32 hash of filename + module type - 431e9adc: ... - ... Inside the cache file, there is a header followed by the actual cache content: ``` [uint32_t] code size [uint32_t] code hash [uint32_t] cache size [uint32_t] cache hash ... compile cache content ... ``` When reading the cache file, we'll also check if the code size and code hash match the code that the module loader is loading and whether the cache size and cache hash match the file content read. If they don't match, or if V8 rejects the cache passed, we'll ignore the mismatch cache, and regenerate the cache after compilation succeeds and rewrite it to disk. PR-URL: nodejs#52535 Refs: nodejs#47472 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>

joyeecheung added 2 commits April 14, 2024 23:00

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 15, 2024

joyeecheung added request-ci Add this label to start a Jenkins CI on a PR. and removed lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 15, 2024

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 15, 2024

joyeecheung mentioned this pull request Apr 15, 2024

Support for file-system based persistent code cache in user-land module loaders #47472

Closed

joyeecheung commented Apr 15, 2024

View reviewed changes

richardlau added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 15, 2024

anonrig reviewed Apr 15, 2024

View reviewed changes

Comment thread src/compile_cache.cc Outdated

VoltrexKeyva reviewed Apr 15, 2024

View reviewed changes

Comment thread src/compile_cache.cc Outdated

benjamingr reviewed Apr 16, 2024

View reviewed changes

benjamingr approved these changes Apr 16, 2024

View reviewed changes

benjamingr reviewed Apr 16, 2024

View reviewed changes

joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024

fixup! module: implement NODE_COMPILE_CACHE for automatic on-disk cod…

b45062b

…e caching

joyeecheung force-pushed the node-compile-cache branch from 9a7e1ed to b45062b Compare April 17, 2024 02:44

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2024

joyeecheung added the notable-change PRs with changes that should be highlighted in changelogs. label Apr 17, 2024

joyeecheung mentioned this pull request Apr 25, 2024

Tracking issue: compile cache #52696

Open

13 tasks

aduh95 added the backported-to-v22.x PRs backported to the v22.x-staging branch. label Apr 30, 2024

aduh95 mentioned this pull request Apr 30, 2024

2024-05-02, Version 22.1.0 (Current) #52768

Merged

marco-ippolito added the backport-blocked-v20.x label May 23, 2024

joyeecheung mentioned this pull request Jun 29, 2024

APIs for libraries/frameworks/tools to control on-disk compilation cache (NODE_COMPILE_CACHE) #53639

Closed

joyeecheung mentioned this pull request Aug 29, 2024

src: add JS APIs for compile cache and NODE_DISABLE_COMPILE_CACHE #54501

Merged

RafaelGSS mentioned this pull request Oct 11, 2024

2024-10-16, Version 23.0.0 (Current) #55338

Merged

joyeecheung mentioned this pull request Feb 7, 2025

[v20.x] backport unflagging of require(esm) to v20 #56927

Closed

+              //     - <cache_file_1>: a hash of filename + module type
+              //     - <cache_file_2>
+              //     - <cache_file_3>
+              bool CompileCacheHandler::InitializeDirectory(const std::string& dir) {

+              of Node.js will be stored separately if the same directory is used
+              to persist the cache, so they can co-exist.
+              Caveat: currently when using this with [V8 JavaScript code coverage][], the


		> Stability: 1.1 - Active Development

		When set, whenever Node.js compiles a CommonJS or a ECMAScript Module,

module: implement NODE_COMPILE_CACHE for automatic on-disk code caching by joyeecheung · Pull Request #52535 · nodejs/node · GitHub

Uh oh!

Uh oh!

module: implement NODE_COMPILE_CACHE for automatic on-disk code caching - #52535

module: implement NODE_COMPILE_CACHE for automatic on-disk code caching#52535joyeecheung wants to merge 3 commits intonodejs:mainnodejs/node:mainfrom joyeecheung:node-compile-cachejoyeecheung/node:node-compile-cacheCopy head branch name to clipboard

Conversation

joyeecheung commented Apr 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

summary

Uh oh!

nodejs-github-bot commented Apr 15, 2024

Uh oh!

nodejs-github-bot commented Apr 15, 2024

Uh oh!

joyeecheung Apr 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

merceyz commented Apr 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

jdalton commented Apr 15, 2024

Uh oh!

Uh oh!

benjamingr Apr 16, 2024

Choose a reason for hiding this comment

Uh oh!

joyeecheung Apr 16, 2024

Choose a reason for hiding this comment

Uh oh!

benjamingr Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

benjamingr left a comment

Choose a reason for hiding this comment

Uh oh!

benjamingr Apr 16, 2024

Choose a reason for hiding this comment

Uh oh!

joyeecheung Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benjamingr Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

joyeecheung Apr 17, 2024

Choose a reason for hiding this comment

Uh oh!

joyeecheung commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

merceyz commented Apr 16, 2024

Uh oh!

joyeecheung commented Apr 16, 2024

Uh oh!

nodejs-github-bot commented Apr 17, 2024

Uh oh!

github-actions Bot commented Apr 17, 2024 • edited by joyeecheung Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lirantal commented Apr 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

module: implement NODE_COMPILE_CACHE for automatic on-disk code caching#52535
joyeecheung wants to merge 3 commits into
nodejs:mainfrom
joyeecheung:node-compile-cache

joyeecheung commented Apr 15, 2024 •
edited
Loading

joyeecheung Apr 15, 2024 •
edited
Loading

merceyz commented Apr 15, 2024 •
edited
Loading

joyeecheung Apr 16, 2024 •
edited
Loading

joyeecheung commented Apr 16, 2024 •
edited
Loading

github-actions Bot commented Apr 17, 2024 •
edited by joyeecheung
Loading