FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

src: avoid deferred gc/cleanup for Buffer.from by jasnell · Pull Request #38337 · nodejs/node · GitHub

/ node Public

src: avoid deferred gc/cleanup for Buffer.from - #38337

Closed
jasnell wants to merge 2 commits into
nodejs:masterfrom
jasnell:avoid-deferred-cleanup-for-buffer-fro
ConversationCommits (2)ChecksFiles changed
Closed

src: avoid deferred gc/cleanup for Buffer.from#38337
jasnell wants to merge 2 commits into
nodejs:masterfrom
jasnell:avoid-deferred-cleanup-for-buffer-fro

Conversation

jasnell commented Apr 21, 2021

Copy link
Copy Markdown
Member

/cc @addaleax @Daninet

Previously, the code path would allocated a tracked ArrayBuffer that defers cleanup and deallocation of the underlying data with a native SetImmediate. Avoid the unnecessary deferral by just allocating a BackingStore directly and writing into it.

This should be considered an alternative to #38336

Fixes: #38300
Refs: #38336

src: avoid deferred gc/cleanup for Buffer.from
6805449 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: nodejs#38300
Refs: nodejs#38336
nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Apr 21, 2021

This comment has been minimized.

Sign in to view
jasnell mentioned this pull request Apr 21, 2021
Closed
addaleax approved these changes Apr 22, 2021
View reviewed changes
Comment thread src/node_buffer.cc Outdated
[Squash] nit
f805a1e 
Co-authored-by: Anna Henningsen <anna@addaleax.net>
jasnell mentioned this pull request Apr 22, 2021
Closed
XadillaX approved these changes Apr 22, 2021
View reviewed changes

XadillaX left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

After testing:

const str = 'x'.repeat(10000);

console.log('start', process.memoryUsage());

console.time('1');
  for(let i = 0; i < 1e6; i++) {
    Buffer.from(str); // memory leak happens here
    if (i % 1e5 === 0) {
      // await wait(100);
      gc();
      console.log('step', i, 'rss', process.memoryUsage().rss);
    }
  }
console.timeEnd('1');

  gc();
  console.log('end', process.memoryUsage());

The new PR #38337 runs about 3.9s in my computer. And 39f42d1a2d (before using BackingStore) runs about 3.7s. I think this performance loss is acceptable.

mcollina approved these changes Apr 22, 2021
View reviewed changes

mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

lgtm

This comment has been minimized.

Sign in to view

This comment has been minimized.

Sign in to view
jasnell added the author ready PRs that have at least one approval, no outstanding review comments, and a CI started. label Apr 22, 2021

This comment has been minimized.

Sign in to view

nodejs-github-bot commented Apr 23, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/37635/

jasnell commented Apr 23, 2021

Copy link
Copy Markdown
Member Author

@nodejs/build ... is there a known issue with the node-test-commit-arm-fanned cluster? It is failing consistently on multiple PRs

richardlau commented Apr 23, 2021

Copy link
Copy Markdown
Member

@nodejs/build ... is there a known issue with the node-test-commit-arm-fanned cluster? It is failing consistently on multiple PRs

I cleaned up leftover node process on three of the pi 2's this morning. AFAIK things should be okay now.

nodejs-github-bot commented Apr 23, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/37643/

jasnell commented Apr 23, 2021

Copy link
Copy Markdown
Member Author

@richardlau ... no joy it seems. See https://ci.nodejs.org/job/node-test-binary-arm-12+/10459/RUN_SUBSET=1,label=pi2-docker/console for #38331, for instance.

targos pushed a commit that referenced this pull request Apr 24, 2021
src: avoid deferred gc/cleanup for Buffer.from
720fdf2 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: #38300
Refs: #38336

PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

targos commented Apr 24, 2021

Copy link
Copy Markdown
Member

Landed in 720fdf2

targos closed this Apr 24, 2021
targos pushed a commit that referenced this pull request Apr 29, 2021
src: avoid deferred gc/cleanup for Buffer.from
3649ec5 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: #38300
Refs: #38336

PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos mentioned this pull request May 3, 2021
Merged
targos pushed a commit that referenced this pull request May 30, 2021
src: avoid deferred gc/cleanup for Buffer.from
c0aa3f0 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: #38300
Refs: #38336

PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos pushed a commit that referenced this pull request Jun 5, 2021
src: avoid deferred gc/cleanup for Buffer.from
ebf79c6 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: #38300
Refs: #38336

PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos mentioned this pull request Jun 6, 2021
Merged
targos pushed a commit that referenced this pull request Jun 11, 2021
src: avoid deferred gc/cleanup for Buffer.from
e279b02 
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.

Fixes: #38300
Refs: #38336

PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

mcollina mcollina approved these changes
addaleax addaleax approved these changes
joyeecheung joyeecheung approved these changes
+1 more reviewer
XadillaX XadillaX approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author ready PRs that have at least one approval, no outstanding review comments, and a CI started. buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Buffer.from(str) memory leak

8 participants

Back | FazBrowse Home | New Git URL