FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

v8: add v8.startupSnapshot utils · nodejs/node@34e8331 · GitHub

/ node Public

Commit 34e8331

Browse files
joyeecheung
authored and
targos
committed
v8: add v8.startupSnapshot utils
This adds several APIs under the `v8.startupSnapshot` namespace for specifying hooks into the startup snapshot serialization and deserialization. - isBuildingSnapshot() - addSerializeCallback() - addDeserializeCallback() - setDeserializeMainFunction() PR-URL: #43329 Fixes: #42617 Refs: #35711 Reviewed-By: Chengzhong Wu <legendecas@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>
1 parent a1f581a commit 34e8331

15 files changed

Lines changed: 413 additions & 29 deletions

File tree

‎doc/api/errors.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1179,6 +1179,13 @@ because the `node:domain` module has been loaded at an earlier point in time.
11791179
The stack trace is extended to include the point in time at which the
11801180
`node:domain` module had been loaded.
11811181

1182+
<a id="ERR_DUPLICATE_STARTUP_SNAPSHOT_MAIN_FUNCTION"></a>
1183+
1184+
### `ERR_DUPLICATE_STARTUP_SNAPSHOT_MAIN_FUNCTION`
1185+
1186+
[`v8.startupSnapshot.setDeserializeMainFunction()`][] could not be called
1187+
because it had already been called before.
1188+
11821189
<a id="ERR_ENCODING_INVALID_ENCODED_DATA"></a>
11831190

11841191
### `ERR_ENCODING_INVALID_ENCODED_DATA`
@@ -2314,6 +2321,13 @@ has occurred when attempting to start the loop.
23142321
Once no more items are left in the queue, the idle loop must be suspended. This
23152322
error indicates that the idle loop has failed to stop.
23162323

2324+
<a id="ERR_NOT_BUILDING_SNAPSHOT"></a>
2325+
2326+
### `ERR_NOT_BUILDING_SNAPSHOT`
2327+
2328+
An attempt was made to use operations that can only be used when building
2329+
V8 startup snapshot even though Node.js isn't building one.
2330+
23172331
<a id="ERR_NO_CRYPTO"></a>
23182332

23192333
### `ERR_NO_CRYPTO`
@@ -3501,6 +3515,7 @@ The native call from `process.cpuUsage` could not be processed.
35013515
[`url.parse()`]: url.md#urlparseurlstring-parsequerystring-slashesdenotehost
35023516
[`util.getSystemErrorName(error.errno)`]: util.md#utilgetsystemerrornameerr
35033517
[`util.parseArgs()`]: util.md#utilparseargsconfig
3518+
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
35043519
[`zlib`]: zlib.md
35053520
[crypto digest algorithm]: crypto.md#cryptogethashes
35063521
[debugger]: debugger.md

‎doc/api/v8.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -876,6 +876,137 @@ Called immediately after a promise continuation executes. This may be after a
876876
Called when the promise receives a resolution or rejection value. This may
877877
occur synchronously in the case of `Promise.resolve()` or `Promise.reject()`.
878878

879+
## Startup Snapshot API
880+
881+
<!-- YAML
882+
added: REPLACEME
883+
-->
884+
885+
> Stability: 1 - Experimental
886+
887+
The `v8.startupSnapshot` interface can be used to add serialization and
888+
deserialization hooks for custom startup snapshots. Currently the startup
889+
snapshots can only be built into the Node.js binary from source.
890+
891+
```console
892+
$ cd /path/to/node
893+
$ ./configure --node-snapshot-main=entry.js
894+
$ make node
895+
# This binary contains the result of the execution of entry.js
896+
$ out/Release/node
897+
```
898+
899+
In the example above, `entry.js` can use methods from the `v8.startupSnapshot`
900+
interface to specify how to save information for custom objects in the snapshot
901+
during serialization and how the information can be used to synchronize these
902+
objects during deserialization of the snapshot. For example, if the `entry.js`
903+
contains the following script:
904+
905+
```cjs
906+
'use strict';
907+
908+
const fs = require('fs');
909+
const zlib = require('zlib');
910+
const path = require('path');
911+
const assert = require('assert');
912+
913+
const {
914+
isBuildingSnapshot,
915+
addSerializeCallback,
916+
addDeserializeCallback,
917+
setDeserializeMainFunction
918+
} = require('v8').startupSnapshot;
919+
920+
const filePath = path.resolve(__dirname, '../x1024.txt');
921+
const storage = {};
922+
923+
assert(isBuildingSnapshot());
924+
925+
addSerializeCallback(({ filePath }) => {
926+
storage[filePath] = zlib.gzipSync(fs.readFileSync(filePath));
927+
}, { filePath });
928+
929+
addDeserializeCallback(({ filePath }) => {
930+
storage[filePath] = zlib.gunzipSync(storage[filePath]);
931+
}, { filePath });
932+
933+
setDeserializeMainFunction(({ filePath }) => {
934+
console.log(storage[filePath].toString());
935+
}, { filePath });
936+
```
937+
938+
The resulted binary will simply print the data deserialized from the snapshot
939+
during start up:
940+
941+
```console
942+
$ out/Release/node
943+
# Prints content of ./test/fixtures/x1024.txt
944+
```
945+
946+
Currently the API is only available to a Node.js instance launched from the
947+
default snapshot, that is, the application deserialized from a user-land
948+
snapshot cannot use these APIs again.
949+
950+
### `v8.startupSnapshot.addSerializeCallback(callback[, data])`
951+
952+
<!-- YAML
953+
added: REPLACEME
954+
-->
955+
956+
* `callback` {Function} Callback to be invoked before serialization.
957+
* `data` {any} Optional data that will be passed to the `callback` when it
958+
gets called.
959+
960+
Add a callback that will be called when the Node.js instance is about to
961+
get serialized into a snapshot and exit. This can be used to release
962+
resources that should not or cannot be serialized or to convert user data
963+
into a form more suitable for serialization.
964+
965+
### `v8.startupSnapshot.addDeserializeCallback(callback[, data])`
966+
967+
<!-- YAML
968+
added: REPLACEME
969+
-->
970+
971+
* `callback` {Function} Callback to be invoked after the snapshot is
972+
deserialized.
973+
* `data` {any} Optional data that will be passed to the `callback` when it
974+
gets called.
975+
976+
Add a callback that will be called when the Node.js instance is deserialized
977+
from a snapshot. The `callback` and the `data` (if provided) will be
978+
serialized into the snapshot, they can be used to re-initialize the state
979+
of the application or to re-acquire resources that the application needs
980+
when the application is restarted from the snapshot.
981+
982+
### `v8.startupSnapshot.setDeserializeMainFunction(callback[, data])`
983+
984+
<!-- YAML
985+
added: REPLACEME
986+
-->
987+
988+
* `callback` {Function} Callback to be invoked as the entry point after the
989+
snapshot is deserialized.
990+
* `data` {any} Optional data that will be passed to the `callback` when it
991+
gets called.
992+
993+
This sets the entry point of the Node.js application when it is deserialized
994+
from a snapshot. This can be called only once in the snapshot building
995+
script. If called, the deserialized application no longer needs an additional
996+
entry point script to start up and will simply invoke the callback along with
997+
the deserialized data (if provided), otherwise an entry point script still
998+
needs to be provided to the deserialized application.
999+
1000+
### `v8.startupSnapshot.isBuildingSnapshot()`
1001+
1002+
<!-- YAML
1003+
added: REPLACEME
1004+
-->
1005+
1006+
* Returns: {boolean}
1007+
1008+
Returns true if the Node.js instance is run to build a snapshot.
1009+
8791010
[HTML structured clone algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
8801011
[Hook Callbacks]: #hook-callbacks
8811012
[V8]: https://developers.google.com/v8/

‎lib/internal/bootstrap/pre_execution.js‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,6 @@ function prepareMainThreadExecution(expandArgv1 = false,
5353
setupCoverageHooks(process.env.NODE_V8_COVERAGE);
5454
}
5555

56-
5756
setupDebugEnv();
5857

5958
// Print stack trace on `SIGINT` if option `--trace-sigint` presents.
@@ -84,6 +83,8 @@ function prepareMainThreadExecution(expandArgv1 = false,
8483
initializeDeprecations();
8584
initializeWASI();
8685

86+
require('internal/v8/startup_snapshot').runDeserializeCallbacks();
87+
8788
if (!initialzeModules) {
8889
return;
8990
}

‎lib/internal/bootstrap/switches/is_main_thread.js‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@
22

33
const { ObjectDefineProperty } = primordials;
44
const rawMethods = internalBinding('process_methods');
5-
5+
const {
6+
addSerializeCallback,
7+
isBuildingSnapshot
8+
} = require('v8').startupSnapshot;
69
// TODO(joyeecheung): deprecate and remove these underscore methods
710
process._debugProcess = rawMethods._debugProcess;
811
process._debugEnd = rawMethods._debugEnd;
@@ -134,6 +137,12 @@ function refreshStderrOnSigWinch() {
134137
stderr._refreshSize();
135138
}
136139

140+
function addCleanup(fn) {
141+
if (isBuildingSnapshot()) {
142+
addSerializeCallback(fn);
143+
}
144+
}
145+
137146
function getStdout() {
138147
if (stdout) return stdout;
139148
stdout = createWritableStdioStream(1);
@@ -145,12 +154,14 @@ function getStdout() {
145154
process.on('SIGWINCH', refreshStdoutOnSigWinch);
146155
}
147156

148-
internalBinding('mksnapshot').cleanups.push(function cleanupStdout() {
157+
addCleanup(function cleanupStdout() {
149158
stdout._destroy = stdoutDestroy;
150159
stdout.destroy();
151160
process.removeListener('SIGWINCH', refreshStdoutOnSigWinch);
152161
stdout = undefined;
153162
});
163+
// No need to add deserialize callback because stdout = undefined above
164+
// causes the stream to be lazily initialized again later.
154165
return stdout;
155166
}
156167

@@ -164,12 +175,14 @@ function getStderr() {
164175
if (stderr.isTTY) {
165176
process.on('SIGWINCH', refreshStderrOnSigWinch);
166177
}
167-
internalBinding('mksnapshot').cleanups.push(function cleanupStderr() {
178+
addCleanup(function cleanupStderr() {
168179
stderr._destroy = stderrDestroy;
169180
stderr.destroy();
170181
process.removeListener('SIGWINCH', refreshStderrOnSigWinch);
171182
stderr = undefined;
172183
});
184+
// No need to add deserialize callback because stderr = undefined above
185+
// causes the stream to be lazily initialized again later.
173186
return stderr;
174187
}
175188

@@ -256,10 +269,12 @@ function getStdin() {
256269
}
257270
}
258271

259-
internalBinding('mksnapshot').cleanups.push(function cleanupStdin() {
272+
addCleanup(function cleanupStdin() {
260273
stdin.destroy();
261274
stdin = undefined;
262275
});
276+
// No need to add deserialize callback because stdin = undefined above
277+
// causes the stream to be lazily initialized again later.
263278
return stdin;
264279
}
265280

‎lib/internal/errors.js‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -978,6 +978,8 @@ E('ERR_DOMAIN_CANNOT_SET_UNCAUGHT_EXCEPTION_CAPTURE',
978978
'The `domain` module is in use, which is mutually exclusive with calling ' +
979979
'process.setUncaughtExceptionCaptureCallback()',
980980
Error);
981+
E('ERR_DUPLICATE_STARTUP_SNAPSHOT_MAIN_FUNCTION',
982+
'Deserialize main function is already configured.', Error);
981983
E('ERR_ENCODING_INVALID_ENCODED_DATA', function(encoding, ret) {
982984
this.errno = ret;
983985
return `The encoded data was not valid for encoding ${encoding}`;
@@ -1456,6 +1458,8 @@ E('ERR_NETWORK_IMPORT_BAD_RESPONSE',
14561458
"import '%s' received a bad response: %s", Error);
14571459
E('ERR_NETWORK_IMPORT_DISALLOWED',
14581460
"import of '%s' by %s is not supported: %s", Error);
1461+
E('ERR_NOT_BUILDING_SNAPSHOT',
1462+
'Operation cannot be invoked when not building startup snapshot', Error);
14591463
E('ERR_NO_CRYPTO',
14601464
'Node.js is not compiled with OpenSSL crypto support', Error);
14611465
E('ERR_NO_ICU',

‎lib/internal/main/mksnapshot.js‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ const {
99
const binding = internalBinding('mksnapshot');
1010
const { NativeModule } = require('internal/bootstrap/loaders');
1111
const {
12-
compileSnapshotMain,
12+
compileSerializeMain,
1313
} = binding;
1414

1515
const {
@@ -83,7 +83,7 @@ const supportedModules = new SafeSet(new SafeArrayIterator([
8383
'v8',
8484
// 'vm',
8585
// 'worker_threads',
86-
// 'zlib',
86+
'zlib',
8787
]));
8888

8989
const warnedModules = new SafeSet();
@@ -117,25 +117,22 @@ function main() {
117117
} = require('internal/bootstrap/pre_execution');
118118

119119
prepareMainThreadExecution(true, false);
120-
process.once('beforeExit', function runCleanups() {
121-
for (const cleanup of binding.cleanups) {
122-
cleanup();
123-
}
124-
});
125120

126121
const file = process.argv[1];
127122
const path = require('path');
128123
const filename = path.resolve(file);
129124
const dirname = path.dirname(filename);
130125
const source = readFileSync(file, 'utf-8');
131-
const snapshotMainFunction = compileSnapshotMain(filename, source);
126+
const serializeMainFunction = compileSerializeMain(filename, source);
127+
128+
require('internal/v8/startup_snapshot').initializeCallbacks();
132129

133130
if (getOptionValue('--inspect-brk')) {
134131
internalBinding('inspector').callAndPauseOnStart(
135-
snapshotMainFunction, undefined,
132+
serializeMainFunction, undefined,
136133
requireForUserSnapshot, filename, dirname);
137134
} else {
138-
snapshotMainFunction(requireForUserSnapshot, filename, dirname);
135+
serializeMainFunction(requireForUserSnapshot, filename, dirname);
139136
}
140137
}
141138

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL