FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

module: fix directory option in the enableCompileCache() API · nodejs/node@31a1c95 · GitHub

/ node Public

Commit 31a1c95

Browse files
joyeecheung
authored and
aduh95
committed
module: fix directory option in the enableCompileCache() API
The option name should be `directory` to be consistent with the returned result. It should also allow environment variable overrides. This also adds documentation for the new options and improves it. PR-URL: #59931 Reviewed-By: Aditi Singh <aditisingh1400@gmail.com>
1 parent c88ea78 commit 31a1c95

5 files changed

Lines changed: 186 additions & 62 deletions

File tree

‎doc/api/module.md‎

Lines changed: 49 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -385,20 +385,33 @@ changes:
385385
386386
The module compile cache can be enabled either using the [`module.enableCompileCache()`][]
387387
method or the [`NODE_COMPILE_CACHE=dir`][] environment variable. After it is enabled,
388-
whenever Node.js compiles a CommonJS or a ECMAScript Module, it will use on-disk
389-
[V8 code cache][] persisted in the specified directory to speed up the compilation.
388+
whenever Node.js compiles a CommonJS, a ECMAScript Module, or a TypeScript module, it will
389+
use on-disk [V8 code cache][] persisted in the specified directory to speed up the compilation.
390390
This may slow down the first load of a module graph, but subsequent loads of the same module
391391
graph may get a significant speedup if the contents of the modules do not change.
392392
393393
To clean up the generated compile cache on disk, simply remove the cache directory. The cache
394394
directory will be recreated the next time the same directory is used for for compile cache
395395
storage. To avoid filling up the disk with stale cache, it is recommended to use a directory
396396
under the [`os.tmpdir()`][]. If the compile cache is enabled by a call to
397-
[`module.enableCompileCache()`][] without specifying the directory, Node.js will use
397+
[`module.enableCompileCache()`][] without specifying the `directory`, Node.js will use
398398
the [`NODE_COMPILE_CACHE=dir`][] environment variable if it's set, or defaults
399399
to `path.join(os.tmpdir(), 'node-compile-cache')` otherwise. To locate the compile cache
400400
directory used by a running Node.js instance, use [`module.getCompileCacheDir()`][].
401401
402+
The enabled module compile cache can be disabled by the [`NODE_DISABLE_COMPILE_CACHE=1`][]
403+
environment variable. This can be useful when the compile cache leads to unexpected or
404+
undesired behaviors (e.g. less precise test coverage).
405+
406+
At the moment, when the compile cache is enabled and a module is loaded afresh, the
407+
code cache is generated from the compiled code immediately, but will only be written
408+
to disk when the Node.js instance is about to exit. This is subject to change. The
409+
[`module.flushCompileCache()`][] method can be used to ensure the accumulated code cache
410+
is flushed to disk in case the application wants to spawn other Node.js instances
411+
and let them share the cache long before the parent exits.
412+
413+
### Portability of the compile cache
414+
402415
By default, caches are invalidated when the absolute paths of the modules being
403416
cached are changed. To keep the cache working after moving the
404417
project directory, enable portable compile cache. This allows previously compiled
@@ -409,38 +422,29 @@ will not be cached.
409422
410423
There are two ways to enable the portable mode:
411424
412-
1. Using the portable option in module.enableCompileCache():
425+
1. Using the portable option in [`module.enableCompileCache()`][]:
413426
414427
```js
415428
// Non-portable cache (default): cache breaks if project is moved
416-
module.enableCompileCache({ path: '/path/to/cache/storage/dir' });
429+
module.enableCompileCache({ directory: '/path/to/cache/storage/dir' });
417430

418431
// Portable cache: cache works after the project is moved
419-
module.enableCompileCache({ path: '/path/to/cache/storage/dir', portable: true });
432+
module.enableCompileCache({ directory: '/path/to/cache/storage/dir', portable: true });
420433
```
421434
422435
2. Setting the environment variable: [`NODE_COMPILE_CACHE_PORTABLE=1`][]
423436
437+
### Limitations of the compile cache
438+
424439
Currently when using the compile cache with [V8 JavaScript code coverage][], the
425440
coverage being collected by V8 may be less precise in functions that are
426441
deserialized from the code cache. It's recommended to turn this off when
427442
running tests to generate precise coverage.
428443
429-
The enabled module compile cache can be disabled by the [`NODE_DISABLE_COMPILE_CACHE=1`][]
430-
environment variable. This can be useful when the compile cache leads to unexpected or
431-
undesired behaviors (e.g. less precise test coverage).
432-
433444
Compilation cache generated by one version of Node.js can not be reused by a different
434445
version of Node.js. Cache generated by different versions of Node.js will be stored
435446
separately if the same base directory is used to persist the cache, so they can co-exist.
436447
437-
At the moment, when the compile cache is enabled and a module is loaded afresh, the
438-
code cache is generated from the compiled code immediately, but will only be written
439-
to disk when the Node.js instance is about to exit. This is subject to change. The
440-
[`module.flushCompileCache()`][] method can be used to ensure the accumulated code cache
441-
is flushed to disk in case the application wants to spawn other Node.js instances
442-
and let them share the cache long before the parent exits.
443-
444448
### `module.constants.compileCacheStatus`
445449
446450
<!-- YAML
@@ -494,16 +498,30 @@ The following constants are returned as the `status` field in the object returne
494498
</tr>
495499
</table>
496500
497-
### `module.enableCompileCache([cacheDir])`
501+
### `module.enableCompileCache([options])`
498502
499503
<!-- YAML
500504
added: v22.8.0
505+
changes:
506+
- version: REPLACEME
507+
pr-url: https://github.com/nodejs/node/pull/58797
508+
description: Add `portable` option to enable portable compile cache.
509+
- version: REPLACEME
510+
pr-url: https://github.com/nodejs/node/pull/59931
511+
description: Rename the unreleased `path` option to `directory` to maintain consistency.
501512
-->
502513
503514
> Stability: 1.1 - Active Development
504515
505-
* `cacheDir` {string|undefined} Optional path to specify the directory where the compile cache
506-
will be stored/retrieved.
516+
* `options` {string|Object} Optional. If a string is passed, it is considered to be `options.directory`.
517+
* `directory` {string} Optional. Directory to store the compile cache. If not specified,
518+
the directory specified by the [`NODE_COMPILE_CACHE=dir`][] environment variable
519+
will be used if it's set, or `path.join(os.tmpdir(), 'node-compile-cache')`
520+
otherwise.
521+
* `portable` {boolean} Optional. If `true`, enables portable compile cache so that
522+
the cache can be reused even if the project directory is moved. This is a best-effort
523+
feature. If not specified, it will depend on whether the environment variable
524+
[`NODE_COMPILE_CACHE_PORTABLE=1`][] is set.
507525
* Returns: {Object}
508526
* `status` {integer} One of the [`module.constants.compileCacheStatus`][]
509527
* `message` {string|undefined} If Node.js cannot enable the compile cache, this contains
@@ -515,20 +533,16 @@ added: v22.8.0
515533
516534
Enable [module compile cache][] in the current Node.js instance.
517535
518-
If `cacheDir` is not specified, Node.js will either use the directory specified by the
519-
[`NODE_COMPILE_CACHE=dir`][] environment variable if it's set, or use
520-
`path.join(os.tmpdir(), 'node-compile-cache')` otherwise. For general use cases, it's
521-
recommended to call `module.enableCompileCache()` without specifying the `cacheDir`,
522-
so that the directory can be overridden by the `NODE_COMPILE_CACHE` environment
523-
variable when necessary.
524-
525-
Since compile cache is supposed to be a quiet optimization that is not required for the
526-
application to be functional, this method is designed to not throw any exception when the
527-
compile cache cannot be enabled. Instead, it will return an object containing an error
528-
message in the `message` field to aid debugging.
529-
If compile cache is enabled successfully, the `directory` field in the returned object
530-
contains the path to the directory where the compile cache is stored. The `status`
531-
field in the returned object would be one of the `module.constants.compileCacheStatus`
536+
For general use cases, it's recommended to call `module.enableCompileCache()` without
537+
specifying the `options.directory`, so that the directory can be overridden by the
538+
`NODE_COMPILE_CACHE` environment variable when necessary.
539+
540+
Since compile cache is supposed to be a optimization that is not mission critical, this
541+
method is designed to not throw any exception when the compile cache cannot be enabled.
542+
Instead, it will return an object containing an error message in the `message` field to
543+
aid debugging. If compile cache is enabled successfully, the `directory` field in the
544+
returned object contains the path to the directory where the compile cache is stored. The
545+
`status` field in the returned object would be one of the `module.constants.compileCacheStatus`
532546
values to indicate the result of the attempt to enable the [module compile cache][].
533547
534548
This method only affects the current Node.js instance. To enable it in child worker threads,
@@ -1817,7 +1831,7 @@ returned object contains the following keys:
18171831
[`SourceMap`]: #class-modulesourcemap
18181832
[`initialize`]: #initialize
18191833
[`module.constants.compileCacheStatus`]: #moduleconstantscompilecachestatus
1820-
[`module.enableCompileCache()`]: #moduleenablecompilecachecachedir
1834+
[`module.enableCompileCache()`]: #moduleenablecompilecacheoptions
18211835
[`module.flushCompileCache()`]: #moduleflushcompilecache
18221836
[`module.getCompileCacheDir()`]: #modulegetcompilecachedir
18231837
[`module.setSourceMapsSupport()`]: #modulesetsourcemapssupportenabled-options

‎lib/internal/modules/helpers.js‎

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -405,28 +405,32 @@ function stringify(body) {
405405
* Enable on-disk compiled cache for all user modules being compiled in the current Node.js instance
406406
* after this method is called.
407407
* This method accepts either:
408-
* - A string `cacheDir`: the path to the cache directory.
409-
* - An options object `{path?: string, portable?: boolean}`:
410-
* - `path`: A string path to the cache directory.
411-
* - `portable`: If `portable` is true, the cache directory will be considered relative. Defaults to false.
412-
* If cache path is undefined, it defaults to the NODE_MODULE_CACHE environment variable.
413-
* If `NODE_MODULE_CACHE` isn't set, it defaults to `path.join(os.tmpdir(), 'node-compile-cache')`.
414-
* @param {string | { path?: string, portable?: boolean } | undefined} options
408+
* - A string: path to the cache directory.
409+
* - An options object `{directory?: string, portable?: boolean}`:
410+
* - `directory`: A string path to the cache directory.
411+
* - `portable`: If `portable` is true, the cache directory will be considered relative.
412+
* Defaults to `NODE_COMPILE_CACHE_PORTABLE === '1'`.
413+
* If cache directory is undefined, it defaults to the `NODE_COMPILE_CACHE` environment variable.
414+
* If `NODE_COMPILE_CACHE` isn't set, it defaults to `path.join(os.tmpdir(), 'node-compile-cache')`.
415+
* @param {string | { directory?: string, portable?: boolean } | undefined} options
415416
* @returns {{status: number, message?: string, directory?: string}}
416417
*/
417418
function enableCompileCache(options) {
418-
let cacheDir;
419-
let portable = false;
419+
let portable;
420+
let directory;
420421

421422
if (typeof options === 'object' && options !== null) {
422-
({ path: cacheDir, portable = false } = options);
423+
({ directory, portable } = options);
423424
} else {
424-
cacheDir = options;
425+
directory = options;
425426
}
426-
if (cacheDir === undefined) {
427-
cacheDir = join(lazyTmpdir(), 'node-compile-cache');
427+
if (directory === undefined) {
428+
directory = process.env.NODE_COMPILE_CACHE || join(lazyTmpdir(), 'node-compile-cache');
428429
}
429-
const nativeResult = _enableCompileCache(cacheDir, portable);
430+
if (portable === undefined) {
431+
portable = process.env.NODE_COMPILE_CACHE_PORTABLE === '1';
432+
}
433+
const nativeResult = _enableCompileCache(directory, portable);
430434
const result = { status: nativeResult[0] };
431435
if (nativeResult[1]) {
432436
result.message = nativeResult[1];

‎test/fixtures/compile-cache-wrapper-options.js‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
'use strict';
2+
3+
const { enableCompileCache, getCompileCacheDir, constants } = require('module');
4+
5+
console.log('dir before enableCompileCache:', getCompileCacheDir());
6+
const options = JSON.parse(process.env.NODE_TEST_COMPILE_CACHE_OPTIONS);
7+
console.log('options:', options);
8+
const result = enableCompileCache(options);
9+
switch (result.status) {
10+
case constants.compileCacheStatus.FAILED:
11+
console.log('Compile cache failed. ' + result.message);
12+
break;
13+
case constants.compileCacheStatus.ENABLED:
14+
console.log('Compile cache enabled. ' + result.directory);
15+
break;
16+
case constants.compileCacheStatus.ALREADY_ENABLED:
17+
console.log('Compile cache already enabled.');
18+
break;
19+
case constants.compileCacheStatus.DISABLED:
20+
console.log('Compile cache already disabled.');
21+
break;
22+
}
23+
console.log('dir after enableCompileCache:', getCompileCacheDir());

‎test/parallel/test-compile-cache-api-options-portable-env.js‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
'use strict';
2+
3+
// This tests module.enableCompileCache() with an options object still works with environment
4+
// variable NODE_COMPILE_CACHE overrides.
5+
6+
require('../common');
7+
const { spawnSyncAndAssert } = require('../common/child_process');
8+
const assert = require('assert');
9+
const fixtures = require('../common/fixtures');
10+
const tmpdir = require('../common/tmpdir');
11+
const fs = require('fs');
12+
const path = require('path');
13+
14+
const wrapper = fixtures.path('compile-cache-wrapper-options.js');
15+
tmpdir.refresh();
16+
17+
// Create a build directory and copy the entry point file.
18+
const buildDir = tmpdir.resolve('build');
19+
fs.mkdirSync(buildDir);
20+
const entryPoint = path.join(buildDir, 'empty.js');
21+
fs.copyFileSync(fixtures.path('empty.js'), entryPoint);
22+
23+
// Check that the portable option can be overridden by NODE_COMPILE_CACHE_PORTABLE.
24+
// We don't override NODE_COMPILE_CACHE because it will enable the cache before
25+
// the wrapper is loaded.
26+
spawnSyncAndAssert(
27+
process.execPath,
28+
['-r', wrapper, entryPoint],
29+
{
30+
env: {
31+
...process.env,
32+
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
33+
NODE_COMPILE_CACHE_PORTABLE: '1',
34+
NODE_TEST_COMPILE_CACHE_OPTIONS: JSON.stringify({ directory: 'build/.compile_cache' }),
35+
},
36+
cwd: tmpdir.path
37+
},
38+
{
39+
stdout(output) {
40+
console.log(output); // Logging for debugging.
41+
assert.match(output, /dir before enableCompileCache: undefined/);
42+
assert.match(output, /Compile cache enabled/);
43+
assert.match(output, /dir after enableCompileCache: .*build[/\\]\.compile_cache/);
44+
return true;
45+
},
46+
stderr(output) {
47+
console.log(output); // Logging for debugging.
48+
assert.match(output, /reading cache from .*build[/\\]\.compile_cache.* for CommonJS .*empty\.js/);
49+
assert.match(output, /empty\.js was not initialized, initializing the in-memory entry/);
50+
assert.match(output, /writing cache for .*empty\.js.*success/);
51+
return true;
52+
}
53+
});
54+
55+
assert(fs.existsSync(tmpdir.resolve('build/.compile_cache')));
56+
57+
const movedDir = buildDir + '_moved';
58+
fs.renameSync(buildDir, movedDir);
59+
const movedEntryPoint = path.join(movedDir, 'empty.js');
60+
61+
// When portable is undefined, it should use the env var NODE_COMPILE_CACHE_PORTABLE.
62+
spawnSyncAndAssert(
63+
process.execPath,
64+
['-r', wrapper, movedEntryPoint],
65+
{
66+
env: {
67+
...process.env,
68+
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
69+
NODE_COMPILE_CACHE_PORTABLE: '1',
70+
NODE_TEST_COMPILE_CACHE_OPTIONS: JSON.stringify({ directory: 'build_moved/.compile_cache' }),
71+
},
72+
cwd: tmpdir.path
73+
},
74+
{
75+
stdout(output) {
76+
console.log(output); // Logging for debugging.
77+
assert.match(output, /dir before enableCompileCache: undefined/);
78+
assert.match(output, /Compile cache enabled/);
79+
assert.match(output, /dir after enableCompileCache: .*build_moved[/\\]\.compile_cache/);
80+
return true;
81+
},
82+
stderr(output) {
83+
console.log(output); // Logging for debugging.
84+
assert.match(output, /reading cache from .*build_moved[/\\]\.compile_cache.* for CommonJS .*empty\.js/);
85+
assert.match(output, /cache for .*empty\.js was accepted, keeping the in-memory entry/);
86+
assert.match(output, /.*skip .*empty\.js because cache was the same/);
87+
return true;
88+
}
89+
});

‎test/parallel/test-compile-cache-api-portable.js‎

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
'use strict';
22

3-
// This tests module.enableCompileCache({ path, portable: true }) works
3+
// This tests module.enableCompileCache({ directory, portable: true }) works
44
// and supports portable paths across directory relocations.
55

66
require('../common');
@@ -9,26 +9,18 @@ const assert = require('assert');
99
const fs = require('fs');
1010
const tmpdir = require('../common/tmpdir');
1111
const path = require('path');
12+
const fixtures = require('../common/fixtures');
1213

1314
tmpdir.refresh();
1415
const workDir = path.join(tmpdir.path, 'work');
1516
const cacheRel = '.compile_cache_dir';
1617
fs.mkdirSync(workDir, { recursive: true });
1718

18-
const wrapper = path.join(workDir, 'wrapper.js');
19+
const wrapper = fixtures.path('compile-cache-wrapper-options.js');
1920
const target = path.join(workDir, 'target.js');
2021

21-
fs.writeFileSync(
22-
wrapper,
23-
`
24-
const { enableCompileCache, getCompileCacheDir } = require('module');
25-
console.log('dir before enableCompileCache:', getCompileCacheDir());
26-
enableCompileCache({ path: '${cacheRel}', portable: true });
27-
console.log('dir after enableCompileCache:', getCompileCacheDir());
28-
`
29-
);
30-
3122
fs.writeFileSync(target, '');
23+
const NODE_TEST_COMPILE_CACHE_OPTIONS = JSON.stringify({ directory: cacheRel, portable: true });
3224

3325
// First run
3426
{
@@ -39,6 +31,7 @@ fs.writeFileSync(target, '');
3931
env: {
4032
...process.env,
4133
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
34+
NODE_TEST_COMPILE_CACHE_OPTIONS,
4235
},
4336
cwd: workDir,
4437
},
@@ -73,13 +66,14 @@ fs.writeFileSync(target, '');
7366
process.execPath,
7467
[
7568
'-r',
76-
path.join(movedWorkDir, 'wrapper.js'),
69+
wrapper,
7770
path.join(movedWorkDir, 'target.js'),
7871
],
7972
{
8073
env: {
8174
...process.env,
8275
NODE_DEBUG_NATIVE: 'COMPILE_CACHE',
76+
NODE_TEST_COMPILE_CACHE_OPTIONS,
8377
},
8478
cwd: movedWorkDir,
8579
},

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL