FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

esm: add experimental support for addon modules · nodejs/node@cdb631e · GitHub

/ node Public

Commit cdb631e

Browse files
legendecas
authored and
aduh95
committed
esm: add experimental support for addon modules
PR-URL: #55844 Fixes: #40541 Fixes: #55821 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
1 parent 3a3f5c9 commit cdb631e

20 files changed

Lines changed: 331 additions & 26 deletions

‎doc/api/cli.md‎

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,11 @@ Otherwise, the file is loaded using the CommonJS module loader. See
4545

4646
When loading, the [ES module loader][Modules loaders] loads the program
4747
entry point, the `node` command will accept as input only files with `.js`,
48-
`.mjs`, or `.cjs` extensions; and with `.wasm` extensions when
49-
[`--experimental-wasm-modules`][] is enabled.
48+
`.mjs`, or `.cjs` extensions. With the following flags, additional file
49+
extensions are enabled:
50+
51+
* [`--experimental-wasm-modules`][] for files with `.wasm` extension.
52+
* [`--experimental-addon-modules`][] for files with `.node` extension.
5053

5154
## Options
5255

@@ -879,6 +882,16 @@ and `"` are usable.
879882
It is possible to run code containing inline types by passing
880883
[`--experimental-strip-types`][].
881884

885+
### `--experimental-addon-modules`
886+
887+
<!-- YAML
888+
added: REPLACEME
889+
-->
890+
891+
> Stability: 1.0 - Early development
892+
893+
Enable experimental import support for `.node` addons.
894+
882895
### `--experimental-async-context-frame`
883896

884897
<!-- YAML
@@ -3046,6 +3059,7 @@ one is included in the list below.
30463059
* `--enable-source-maps`
30473060
* `--entry-url`
30483061
* `--experimental-abortcontroller`
3062+
* `--experimental-addon-modules`
30493063
* `--experimental-async-context-frame`
30503064
* `--experimental-detect-module`
30513065
* `--experimental-eventsource`
@@ -3617,6 +3631,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
36173631
[`--diagnostic-dir`]: #--diagnostic-dirdirectory
36183632
[`--env-file-if-exists`]: #--env-file-if-existsconfig
36193633
[`--env-file`]: #--env-fileconfig
3634+
[`--experimental-addon-modules`]: #--experimental-addon-modules
36203635
[`--experimental-sea-config`]: single-executable-applications.md#generating-single-executable-preparation-blobs
36213636
[`--experimental-strip-types`]: #--experimental-strip-types
36223637
[`--experimental-wasm-modules`]: #--experimental-wasm-modules

‎doc/api/esm.md‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1039,18 +1039,21 @@ _isImports_, _conditions_)
10391039
> 5. If `--experimental-wasm-modules` is enabled and _url_ ends in
10401040
> _".wasm"_, then
10411041
> 1. Return _"wasm"_.
1042-
> 6. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_url_).
1043-
> 7. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_).
1044-
> 8. Let _packageType_ be **null**.
1045-
> 9. If _pjson?.type_ is _"module"_ or _"commonjs"_, then
1046-
> 1. Set _packageType_ to _pjson.type_.
1047-
> 10. If _url_ ends in _".js"_, then
1042+
> 6. If `--experimental-addon-modules` is enabled and _url_ ends in
1043+
> _".node"_, then
1044+
> 1. Return _"addon"_.
1045+
> 7. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_url_).
1046+
> 8. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_).
1047+
> 9. Let _packageType_ be **null**.
1048+
> 10. If _pjson?.type_ is _"module"_ or _"commonjs"_, then
1049+
> 1. Set _packageType_ to _pjson.type_.
1050+
> 11. If _url_ ends in _".js"_, then
10481051
> 1. If _packageType_ is not **null**, then
10491052
> 1. Return _packageType_.
10501053
> 2. If the result of **DETECT\_MODULE\_SYNTAX**(_source_) is true, then
10511054
> 1. Return _"module"_.
10521055
> 3. Return _"commonjs"_.
1053-
> 11. If _url_ does not have any extension, then
1056+
> 12. If _url_ does not have any extension, then
10541057
> 1. If _packageType_ is _"module"_ and `--experimental-wasm-modules` is
10551058
> enabled and the file at _url_ contains the header for a WebAssembly
10561059
> module, then
@@ -1060,7 +1063,7 @@ _isImports_, _conditions_)
10601063
> 3. If the result of **DETECT\_MODULE\_SYNTAX**(_source_) is true, then
10611064
> 1. Return _"module"_.
10621065
> 4. Return _"commonjs"_.
1063-
> 12. Return **undefined** (will throw during load phase).
1066+
> 13. Return **undefined** (will throw during load phase).
10641067
10651068
**LOOKUP\_PACKAGE\_SCOPE**(_url_)
10661069

‎doc/api/module.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1160,6 +1160,7 @@ The final value of `format` must be one of the following:
11601160
11611161
| `format` | Description | Acceptable types for `source` returned by `load` |
11621162
| ------------ | ------------------------------ | -------------------------------------------------- |
1163+
| `'addon'` | Load a Node.js addon | {null} |
11631164
| `'builtin'` | Load a Node.js builtin module | {null} |
11641165
| `'commonjs'` | Load a Node.js CommonJS module | {string\|ArrayBuffer\|TypedArray\|null\|undefined} |
11651166
| `'json'` | Load a JSON file | {string\|ArrayBuffer\|TypedArray} |

‎doc/node.1‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,9 @@ Enable Source Map V3 support for stack traces.
163163
.It Fl -entry-url
164164
Interpret the entry point as a URL.
165165
.
166+
.It Fl -experimental-addon-modules
167+
Enable experimental addon module support.
168+
.
166169
.It Fl -experimental-import-meta-resolve
167170
Enable experimental ES modules support for import.meta.resolve().
168171
.

‎lib/internal/modules/esm/formats.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ const fsBindings = internalBinding('fs');
1010
const { fs: fsConstants } = internalBinding('constants');
1111

1212
const experimentalWasmModules = getOptionValue('--experimental-wasm-modules');
13+
const experimentalAddonModules = getOptionValue('--experimental-addon-modules');
1314

1415
const extensionFormatMap = {
1516
'__proto__': null,
@@ -23,6 +24,10 @@ if (experimentalWasmModules) {
2324
extensionFormatMap['.wasm'] = 'wasm';
2425
}
2526

27+
if (experimentalAddonModules) {
28+
extensionFormatMap['.node'] = 'addon';
29+
}
30+
2631
if (getOptionValue('--experimental-strip-types')) {
2732
extensionFormatMap['.ts'] = 'module-typescript';
2833
extensionFormatMap['.mts'] = 'module-typescript';

‎lib/internal/modules/esm/load.js‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ async function defaultLoad(url, context = kEmptyObject) {
105105
if (urlInstance.protocol === 'node:') {
106106
source = null;
107107
format ??= 'builtin';
108+
} else if (format === 'addon') {
109+
// Skip loading addon file content. It must be loaded with dlopen from file system.
110+
source = null;
108111
} else if (format !== 'commonjs') {
109112
if (source == null) {
110113
({ responseURL, source } = await getSource(urlInstance, context));

‎lib/internal/modules/esm/translators.js‎

Lines changed: 88 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
const {
44
ArrayPrototypeMap,
55
ArrayPrototypePush,
6-
Boolean,
76
FunctionPrototypeCall,
87
JSONParse,
98
ObjectKeys,
@@ -52,6 +51,7 @@ let debug = require('internal/util/debuglog').debuglog('esm', (fn) => {
5251
});
5352
const { emitExperimentalWarning, kEmptyObject, setOwnProperty, isWindows } = require('internal/util');
5453
const {
54+
ERR_INVALID_RETURN_PROPERTY_VALUE,
5555
ERR_UNKNOWN_BUILTIN_MODULE,
5656
} = require('internal/errors').codes;
5757
const { maybeCacheSourceMap } = require('internal/source_map/source_map_cache');
@@ -185,7 +185,7 @@ function createCJSModuleWrap(url, source, isMain, format, loadCJS = loadCJSModul
185185
// In case the source was not provided by the `load` step, we need fetch it now.
186186
source = stringify(source ?? getSource(new URL(url)).source);
187187

188-
const { exportNames, module } = cjsPreparseModuleExports(filename, source, isMain, format);
188+
const { exportNames, module } = cjsPreparseModuleExports(filename, source, format);
189189
cjsCache.set(url, module);
190190

191191
const wrapperNames = [...exportNames, 'module.exports'];
@@ -229,6 +229,47 @@ function createCJSModuleWrap(url, source, isMain, format, loadCJS = loadCJSModul
229229
}, module);
230230
}
231231

232+
/**
233+
* Creates a ModuleWrap object for a CommonJS module without source texts.
234+
* @param {string} url - The URL of the module.
235+
* @param {boolean} isMain - Whether the module is the main module.
236+
* @returns {ModuleWrap} The ModuleWrap object for the CommonJS module.
237+
*/
238+
function createCJSNoSourceModuleWrap(url, isMain) {
239+
debug(`Translating CJSModule without source ${url}`);
240+
241+
const filename = urlToFilename(url);
242+
243+
const module = cjsEmplaceModuleCacheEntry(filename);
244+
cjsCache.set(url, module);
245+
246+
if (isMain) {
247+
setOwnProperty(process, 'mainModule', module);
248+
}
249+
250+
// Addon export names are not known until the addon is loaded.
251+
const exportNames = ['default', 'module.exports'];
252+
return new ModuleWrap(url, undefined, exportNames, function evaluationCallback() {
253+
debug(`Loading CJSModule ${url}`);
254+
255+
if (!module.loaded) {
256+
wrapModuleLoad(filename, null, isMain);
257+
}
258+
259+
/** @type {import('./loader').ModuleExports} */
260+
let exports;
261+
if (module[kModuleExport] !== undefined) {
262+
exports = module[kModuleExport];
263+
module[kModuleExport] = undefined;
264+
} else {
265+
({ exports } = module);
266+
}
267+
268+
this.setExport('default', exports);
269+
this.setExport('module.exports', exports);
270+
}, module);
271+
}
272+
232273
translators.set('commonjs-sync', function requireCommonJS(url, source, isMain) {
233274
initCJSParseSync();
234275

@@ -280,26 +321,38 @@ translators.set('commonjs', function commonjsStrategy(url, source, isMain) {
280321
return createCJSModuleWrap(url, source, isMain, 'commonjs', cjsLoader);
281322
});
282323

324+
/**
325+
* Get or create an entry in the CJS module cache for the given filename.
326+
* @param {string} filename CJS module filename
327+
* @returns {CJSModule} the cached CJS module entry
328+
*/
329+
function cjsEmplaceModuleCacheEntry(filename, exportNames) {
330+
// TODO: Do we want to keep hitting the user mutable CJS loader here?
331+
let cjsMod = CJSModule._cache[filename];
332+
if (cjsMod) {
333+
return cjsMod;
334+
}
335+
336+
cjsMod = new CJSModule(filename);
337+
cjsMod.filename = filename;
338+
cjsMod.paths = CJSModule._nodeModulePaths(cjsMod.path);
339+
cjsMod[kIsCachedByESMLoader] = true;
340+
CJSModule._cache[filename] = cjsMod;
341+
342+
return cjsMod;
343+
}
344+
283345
/**
284346
* Pre-parses a CommonJS module's exports and re-exports.
285347
* @param {string} filename - The filename of the module.
286348
* @param {string} [source] - The source code of the module.
287-
* @param {boolean} isMain - Whether it is pre-parsing for the entry point.
288-
* @param {string} format
349+
* @param {string} [format]
289350
*/
290-
function cjsPreparseModuleExports(filename, source, isMain, format) {
291-
let module = CJSModule._cache[filename];
292-
if (module && module[kModuleExportNames] !== undefined) {
351+
function cjsPreparseModuleExports(filename, source, format) {
352+
const module = cjsEmplaceModuleCacheEntry(filename);
353+
if (module[kModuleExportNames] !== undefined) {
293354
return { module, exportNames: module[kModuleExportNames] };
294355
}
295-
const loaded = Boolean(module);
296-
if (!loaded) {
297-
module = new CJSModule(filename);
298-
module.filename = filename;
299-
module.paths = CJSModule._nodeModulePaths(module.path);
300-
module[kIsCachedByESMLoader] = true;
301-
CJSModule._cache[filename] = module;
302-
}
303356

304357
if (source === undefined) {
305358
({ source } = loadSourceForCJSWithHooks(module, filename, format));
@@ -340,7 +393,7 @@ function cjsPreparseModuleExports(filename, source, isMain, format) {
340393

341394
if (format === 'commonjs' ||
342395
(!BuiltinModule.normalizeRequirableId(resolved) && findLongestRegisteredExtension(resolved) === '.js')) {
343-
const { exportNames: reexportNames } = cjsPreparseModuleExports(resolved, undefined, false, format);
396+
const { exportNames: reexportNames } = cjsPreparseModuleExports(resolved, undefined, format);
344397
for (const name of reexportNames) {
345398
exportNames.add(name);
346399
}
@@ -462,6 +515,25 @@ translators.set('wasm', async function(url, source) {
462515
}).module;
463516
});
464517

518+
// Strategy for loading a addon
519+
translators.set('addon', function translateAddon(url, source, isMain) {
520+
emitExperimentalWarning('Importing addons');
521+
522+
// The addon must be loaded from file system with dlopen. Assert
523+
// the source is null.
524+
if (source !== null) {
525+
throw new ERR_INVALID_RETURN_PROPERTY_VALUE(
526+
'null',
527+
'load',
528+
'source',
529+
source);
530+
}
531+
532+
debug(`Translating addon ${url}`);
533+
534+
return createCJSNoSourceModuleWrap(url, isMain);
535+
});
536+
465537
// Strategy for loading a commonjs TypeScript module
466538
translators.set('commonjs-typescript', function(url, source) {
467539
emitExperimentalWarning('Type Stripping');

‎src/node_options.cc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -409,6 +409,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
409409
"Treat the entrypoint as a URL",
410410
&EnvironmentOptions::entry_is_url,
411411
kAllowedInEnvvar);
412+
AddOption("--experimental-addon-modules",
413+
"experimental import support for addons",
414+
&EnvironmentOptions::experimental_addon_modules,
415+
kAllowedInEnvvar);
412416
AddOption("--experimental-abortcontroller", "", NoOp{}, kAllowedInEnvvar);
413417
AddOption("--experimental-eventsource",
414418
"experimental EventSource API",

‎src/node_options.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ class EnvironmentOptions : public Options {
120120
bool require_module = true;
121121
std::string dns_result_order;
122122
bool enable_source_maps = false;
123+
bool experimental_addon_modules = false;
123124
bool experimental_eventsource = false;
124125
bool experimental_fetch = true;
125126
bool experimental_websocket = true;

‎test/addons/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ Makefile
55
*.mk
66
gyp-mac-tool
77
/*/build
8+
/esm/node_modules/*/build

0 commit comments

Comments
 (0)
Back | FazBrowse Home | New Git URL