FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

esm: add `initialize` hook, integrate with `register` · nodejs/node@10e7c3a · GitHub

/ node Public

Commit 10e7c3a

Browse files
izaakschroeder
authored and
targos
committed
esm: add initialize hook, integrate with register
Follows @giltayar's proposed API: > `register` can pass any data it wants to the loader, which will be passed to the exported `initialize` function of the loader. Additionally, if the user of `register` wants to communicate with the loader, it can just create a `MessageChannel` and pass the port to the loader as data. The `register` API is now: ```ts interface Options { parentUrl?: string; data?: any; transferList?: any[]; } function register(loader: string, parentUrl?: string): any; function register(loader: string, options?: Options): any; ``` This API is backwards compatible with the old one (new arguments are optional and at the end) and allows for passing data into the new `initialize` hook. If this hook returns data it is passed back to `register`: ```ts function initialize(data: any): Promise<any>; ``` **NOTE**: Currently there is no mechanism for a loader to exchange ownership of something back to the caller. Refs: nodejs/loaders#147 PR-URL: #48842 Backport-PR-URL: #50669 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
1 parent f96b610 commit 10e7c3a

10 files changed

Lines changed: 414 additions & 28 deletions

File tree

‎doc/api/esm.md‎

Lines changed: 71 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -684,6 +684,9 @@ of Node.js applications.
684684
<!-- YAML
685685
added: v8.8.0
686686
changes:
687+
- version: REPLACEME
688+
pr-url: https://github.com/nodejs/node/pull/48842
689+
description: Added `initialize` hook to replace `globalPreload`.
687690
- version:
688691
- v18.6.0
689692
pr-url: https://github.com/nodejs/node/pull/42623
@@ -737,6 +740,69 @@ different [realm](https://tc39.es/ecma262/#realm). The hooks thread may be
737740
terminated by the main thread at any time, so do not depend on asynchronous
738741
operations to (like `console.log`) complete.
739742
743+
#### `initialize()`
744+
745+
<!-- YAML
746+
added: REPLACEME
747+
-->
748+
749+
> The loaders API is being redesigned. This hook may disappear or its
750+
> signature may change. Do not rely on the API described below.
751+
752+
* `data` {any} The data from `register(loader, import.meta.url, { data })`.
753+
* Returns: {any} The data to be returned to the caller of `register`.
754+
755+
The `initialize` hook provides a way to define a custom function that runs
756+
in the loader's thread when the loader is initialized. Initialization happens
757+
when the loader is registered via [`register`][] or registered via the
758+
`--loader` command line option.
759+
760+
This hook can send and receive data from a [`register`][] invocation, including
761+
ports and other transferrable objects. The return value of `initialize` must be
762+
either:
763+
764+
* `undefined`,
765+
* something that can be posted as a message between threads (e.g. the input to
766+
[`port.postMessage`][]),
767+
* a `Promise` resolving to one of the aforementioned values.
768+
769+
Loader code:
770+
771+
```js
772+
// In the below example this file is referenced as
773+
// '/path-to-my-loader.js'
774+
775+
export async function initialize({ number, port }) {
776+
port.postMessage(`increment: ${number + 1}`);
777+
return 'ok';
778+
}
779+
```
780+
781+
Caller code:
782+
783+
```js
784+
import assert from 'node:assert';
785+
import { register } from 'node:module';
786+
import { MessageChannel } from 'node:worker_threads';
787+
788+
// This example showcases how a message channel can be used to
789+
// communicate between the main (application) thread and the loader
790+
// running on the loaders thread, by sending `port2` to the loader.
791+
const { port1, port2 } = new MessageChannel();
792+
793+
port1.on('message', (msg) => {
794+
assert.strictEqual(msg, 'increment: 2');
795+
});
796+
797+
const result = register('/path-to-my-loader.js', {
798+
parentURL: import.meta.url,
799+
data: { number: 1, port: port2 },
800+
transferList: [port2],
801+
});
802+
803+
assert.strictEqual(result, 'ok');
804+
```
805+
740806
#### `resolve(specifier, context, nextResolve)`
741807
742808
<!-- YAML
@@ -941,8 +1007,8 @@ changes:
9411007
description: Add support for chaining globalPreload hooks.
9421008
-->
9431009
944-
> The loaders API is being redesigned. This hook may disappear or its
945-
> signature may change. Do not rely on the API described below.
1010+
> This hook will be removed in a future version. Use [`initialize`][] instead.
1011+
> When a loader has an `initialize` export, `globalPreload` will be ignored.
9461012
9471013
> In a previous version of this API, this hook was named
9481014
> `getGlobalPreloadCode`.
@@ -1642,13 +1708,16 @@ success!
16421708
[`import.meta.resolve`]: #importmetaresolvespecifier-parent
16431709
[`import.meta.url`]: #importmetaurl
16441710
[`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
1711+
[`initialize`]: #initialize
16451712
[`module.createRequire()`]: module.md#modulecreaterequirefilename
16461713
[`module.register()`]: module.md#moduleregister
16471714
[`module.syncBuiltinESMExports()`]: module.md#modulesyncbuiltinesmexports
16481715
[`package.json`]: packages.md#nodejs-packagejson-field-definitions
1716+
[`port.postMessage`]: worker_threads.md#portpostmessagevalue-transferlist
16491717
[`port.ref()`]: https://nodejs.org/dist/latest-v17.x/docs/api/worker_threads.html#portref
16501718
[`port.unref()`]: https://nodejs.org/dist/latest-v17.x/docs/api/worker_threads.html#portunref
16511719
[`process.dlopen`]: process.md#processdlopenmodule-filename-flags
1720+
[`register`]: module.md#moduleregister
16521721
[`string`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String
16531722
[`util.TextDecoder`]: util.md#class-utiltextdecoder
16541723
[cjs-module-lexer]: https://github.com/nodejs/cjs-module-lexer/tree/1.2.2

‎doc/api/module.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,28 @@ globalPreload: http-to-https
173173
globalPreload: unpkg
174174
```
175175
176+
This function can also be used to pass data to the loader's [`initialize`][]
177+
hook; the data passed to the hook may include transferrable objects like ports.
178+
179+
```mjs
180+
import { register } from 'node:module';
181+
import { MessageChannel } from 'node:worker_threads';
182+
183+
// This example showcases how a message channel can be used to
184+
// communicate to the loader, by sending `port2` to the loader.
185+
const { port1, port2 } = new MessageChannel();
186+
187+
port1.on('message', (msg) => {
188+
console.log(msg);
189+
});
190+
191+
register('./my-programmatic-loader.mjs', {
192+
parentURL: import.meta.url,
193+
data: { number: 1, port: port2 },
194+
transferList: [port2],
195+
});
196+
```
197+
176198
### `module.syncBuiltinESMExports()`
177199
178200
<!-- YAML
@@ -358,6 +380,7 @@ returned object contains the following keys:
358380
[`--enable-source-maps`]: cli.md#--enable-source-maps
359381
[`NODE_V8_COVERAGE=dir`]: cli.md#node_v8_coveragedir
360382
[`SourceMap`]: #class-modulesourcemap
383+
[`initialize`]: esm.md#initialize
361384
[`module`]: modules.md#the-module-object
362385
[module wrapper]: modules.md#the-module-wrapper
363386
[source map include directives]: https://sourcemaps.info/spec.html#h.lmz475t4mvbx

‎lib/internal/modules/esm/hooks.js‎

Lines changed: 56 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
const {
44
ArrayPrototypePush,
5+
ArrayPrototypePushApply,
56
FunctionPrototypeCall,
67
Int32Array,
78
ObjectAssign,
@@ -46,8 +47,10 @@ const {
4647
validateObject,
4748
validateString,
4849
} = require('internal/validators');
49-
50-
const { kEmptyObject } = require('internal/util');
50+
const {
51+
emitExperimentalWarning,
52+
kEmptyObject,
53+
} = require('internal/util');
5154

5255
const {
5356
defaultResolve,
@@ -82,6 +85,7 @@ let importMetaInitializer;
8285

8386
// [2] `validate...()`s throw the wrong error
8487

88+
let globalPreloadWarned = false;
8589
class Hooks {
8690
#chains = {
8791
/**
@@ -126,31 +130,43 @@ class Hooks {
126130
* Import and register custom/user-defined module loader hook(s).
127131
* @param {string} urlOrSpecifier
128132
* @param {string} parentURL
133+
* @param {any} [data] Arbitrary data to be passed from the custom
134+
* loader (user-land) to the worker.
129135
*/
130-
async register(urlOrSpecifier, parentURL) {
136+
async register(urlOrSpecifier, parentURL, data) {
131137
const moduleLoader = require('internal/process/esm_loader').esmLoader;
132138
const keyedExports = await moduleLoader.import(
133139
urlOrSpecifier,
134140
parentURL,
135141
kEmptyObject,
136142
);
137-
this.addCustomLoader(urlOrSpecifier, keyedExports);
143+
return this.addCustomLoader(urlOrSpecifier, keyedExports, data);
138144
}
139145

140146
/**
141147
* Collect custom/user-defined module loader hook(s).
142148
* After all hooks have been collected, the global preload hook(s) must be initialized.
143149
* @param {string} url Custom loader specifier
144150
* @param {Record<string, unknown>} exports
151+
* @param {any} [data] Arbitrary data to be passed from the custom loader (user-land)
152+
* to the worker.
153+
* @returns {any} The result of the loader's `initialize` hook, if provided.
145154
*/
146-
addCustomLoader(url, exports) {
155+
addCustomLoader(url, exports, data) {
147156
const {
148157
globalPreload,
158+
initialize,
149159
resolve,
150160
load,
151161
} = pluckHooks(exports);
152162

153-
if (globalPreload) {
163+
if (globalPreload && !initialize) {
164+
if (globalPreloadWarned === false) {
165+
globalPreloadWarned = true;
166+
emitExperimentalWarning(
167+
'`globalPreload` will be removed in a future version. Please use `initialize` instead.',
168+
);
169+
}
154170
ArrayPrototypePush(this.#chains.globalPreload, { __proto__: null, fn: globalPreload, url });
155171
}
156172
if (resolve) {
@@ -161,6 +177,7 @@ class Hooks {
161177
const next = this.#chains.load[this.#chains.load.length - 1];
162178
ArrayPrototypePush(this.#chains.load, { __proto__: null, fn: load, url, next });
163179
}
180+
return initialize?.(data);
164181
}
165182

166183
/**
@@ -552,15 +569,30 @@ class HooksProxy {
552569
}
553570
}
554571

555-
async makeAsyncRequest(method, ...args) {
572+
/**
573+
* Invoke a remote method asynchronously.
574+
* @param {string} method Method to invoke
575+
* @param {any[]} [transferList] Objects in `args` to be transferred
576+
* @param {any[]} args Arguments to pass to `method`
577+
* @returns {Promise<any>}
578+
*/
579+
async makeAsyncRequest(method, transferList, ...args) {
556580
this.waitForWorker();
557581

558582
MessageChannel ??= require('internal/worker/io').MessageChannel;
559583
const asyncCommChannel = new MessageChannel();
560584

561585
// Pass work to the worker.
562-
debug('post async message to worker', { method, args });
563-
this.#worker.postMessage({ method, args, port: asyncCommChannel.port2 }, [asyncCommChannel.port2]);
586+
debug('post async message to worker', { method, args, transferList });
587+
const finalTransferList = [asyncCommChannel.port2];
588+
if (transferList) {
589+
ArrayPrototypePushApply(finalTransferList, transferList);
590+
}
591+
this.#worker.postMessage({
592+
__proto__: null,
593+
method, args,
594+
port: asyncCommChannel.port2,
595+
}, finalTransferList);
564596

565597
if (this.#numberOfPendingAsyncResponses++ === 0) {
566598
// On the next lines, the main thread will await a response from the worker thread that might
@@ -592,12 +624,19 @@ class HooksProxy {
592624
return body;
593625
}
594626

595-
makeSyncRequest(method, ...args) {
627+
/**
628+
* Invoke a remote method synchronously.
629+
* @param {string} method Method to invoke
630+
* @param {any[]} [transferList] Objects in `args` to be transferred
631+
* @param {any[]} args Arguments to pass to `method`
632+
* @returns {any}
633+
*/
634+
makeSyncRequest(method, transferList, ...args) {
596635
this.waitForWorker();
597636

598637
// Pass work to the worker.
599-
debug('post sync message to worker', { method, args });
600-
this.#worker.postMessage({ method, args });
638+
debug('post sync message to worker', { method, args, transferList });
639+
this.#worker.postMessage({ __proto__: null, method, args }, transferList);
601640

602641
let response;
603642
do {
@@ -707,6 +746,7 @@ ObjectSetPrototypeOf(HooksProxy.prototype, null);
707746
*/
708747
function pluckHooks({
709748
globalPreload,
749+
initialize,
710750
resolve,
711751
load,
712752
}) {
@@ -722,6 +762,10 @@ function pluckHooks({
722762
acceptedHooks.load = load;
723763
}
724764

765+
if (initialize) {
766+
acceptedHooks.initialize = initialize;
767+
}
768+
725769
return acceptedHooks;
726770
}
727771

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL