FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

module: refactor to use iterable-weak-map by bcoe · Pull Request #35915 · nodejs/node · GitHub

/ node Public

module: refactor to use iterable-weak-map - #35915

Closed
bcoe wants to merge 10 commits into
nodejs:masterfrom
bcoe:iterable-weak-map
Closed

module: refactor to use iterable-weak-map#35915
bcoe wants to merge 10 commits into
nodejs:masterfrom
bcoe:iterable-weak-map

Conversation

bcoe commented Nov 1, 2020
edited
Loading

Copy link
Copy Markdown
Contributor

Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

nodejs-github-bot commented Nov 1, 2020
edited by bcoe
Loading

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/modules

nodejs-github-bot added build Issues and PRs related to build files or the CI. util Issues and PRs related to the built-in util module. labels Nov 1, 2020
Comment thread lib/internal/util/iterable-weak-map.js Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@joyeecheung @addaleax when I try to define FinalizationRegistry and WeakRef in primordials, I get a compilation error because the WeakRef and FinalizationRegistry are undefined at the time when primordials is initially loaded.

Any thoughts?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

WeakRef and FinalizationRegistry cannot be used with snapshots because they can be disabled using a flag at runtime.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think this means we can't use them for a utility like this?

We don't want source maps to stop working because the flag isn't set.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It's a V8 --harmony flag, that's not supported and undocumented in Node.js. I'd say it's fine to use them.

Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't understand how this works - this appears to create a new WeakRef each time the key is set even if the key "value" already exists in the map? Is this intentional?

benjamingr Nov 1, 2020
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality
var m = new Map();
var o = {};
var r1 = new WeakRef(o);
var r2 = new WeakRef(o);
m.set(r1, "foo");
console.log(m.get(r1)); // foo 
console.log(m.get(r2)); // undefined

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

this appears to create a new WeakRef each time the key is set even if the key "value" already exists in the map? Is this intentional

I'll add unit tests for the IterableWeakMap implementation, and will make sure this case is covered.

benjamingr Nov 2, 2020
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm saying - I don't understand how it works and to my understanding it's not supposed to. A test is fine, but a "this works because X" is also fine :]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@benjamingr I'm pretty sure you've identified a bug in the IterableWeakMap shared on the TC39 spec page -- we should only create a new WeakRef if there's not already an entry, otherwise we should perform an update.

Once I've fixed this, I'll share the updated code, and can provide a walk through of the logic if it's still confusing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Sure, feel free to ping me when updated and I'll take a look :]

Comment thread doc/api/module.md Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated

bcoe commented Nov 2, 2020

Copy link
Copy Markdown
Contributor Author

CC: @mcollina, I noticed a conversation you had on Twitter contemplating using IterableWeakMaps for a similar purpose.

Comment thread lib/internal/util/iterable-weak-map.js Outdated
Comment thread lib/internal/util/iterable-weak-map.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

to call it out explicitly: i think these primordials todos should be considered blockers

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@ljharb @aduh95, how about the approach of exposing the makeSafe from primordials, and freezing WeakRef and FinalizationRegistry when iterable-weak-map.js is loaded:

// TODO(aduh95): Add WeakRef to primordials
const SafeWeakRef = makeSafe(
  globalThis.WeakRef,
  class SafeWeakRef extends globalThis.WeakRef {}
);

// This class is modified from the example code in the WeakRefs specification:
// https://github.com/tc39/proposal-weakrefs
// Licensed under ECMA's MIT-style license, see:
// https://github.com/tc39/ecma262/blob/master/LICENSE.md
class IterableWeakMap {
// ...
}}

source_map_cache.js pretty early on in the application lifecycle, _I think, before any userland code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

That seems fine, but I’m not sure what’s wrong with eagerly freezing it, like all the other primordials?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

When the primordials are prepared, we only have access to a subset of builtins. V8 does not provide us the ones that can be disabled with flags by the user.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

ahhhh thanks, that makes sense.

bcoe changed the title module: refactor to use iterable-weak-map errors: refactor to use iterable-weak-map Nov 3, 2020
Comment thread test/parallel/test-source-map-enable.js Outdated
Comment thread test/parallel/test-source-map-enable.js Outdated
Comment thread lib/internal/util/iterable_weak_map.js Outdated

Copy link
Copy Markdown
Member

Generally look fine, lmk when it's ready and I'll check the branch out and play with it a bit :]

bcoe changed the title errors: refactor to use iterable-weak-map module: refactor to use iterable-weak-map Nov 4, 2020
bcoe marked this pull request as ready for review November 4, 2020 14:38
bcoe requested a review from benjamingr November 4, 2020 14:39

This comment has been minimized.

bcoe requested a review from aduh95 November 4, 2020 14:39
bcoe and others added 9 commits November 5, 2020 07:34
bcoe force-pushed the iterable-weak-map branch from 8ba5609 to f1d088a Compare November 5, 2020 15:43
bcoe requested a review from aduh95 November 5, 2020 15:47
bcoe added the author ready PRs that have at least one approval, no outstanding review comments, and a CI started. label Nov 5, 2020

This comment has been minimized.

Copy link
Copy Markdown
Collaborator

bcoe commented Nov 6, 2020

Copy link
Copy Markdown
Contributor Author

@benjamingr happy with how this is looking?

mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

lgtm

bcoe pushed a commit that referenced this pull request Nov 6, 2020
Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().

PR-URL: #35915
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

bcoe commented Nov 6, 2020

Copy link
Copy Markdown
Contributor Author

Landed in 8fa9035

bcoe closed this Nov 6, 2020
bcoe deleted the iterable-weak-map branch November 6, 2020 20:48
danielleadams pushed a commit that referenced this pull request Nov 9, 2020
Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().

PR-URL: #35915
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
danielleadams mentioned this pull request Nov 9, 2020
targos pushed a commit that referenced this pull request May 16, 2021
Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().

PR-URL: #35915
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no outstanding review comments, and a CI started. build Issues and PRs related to build files or the CI. util Issues and PRs related to the built-in util module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants


Back | FazBrowse Home | New Git URL