FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Ensure async iteration works with destroyed streams by mcollina · Pull Request #23785 · nodejs/node · GitHub

/ node Public

Ensure async iteration works with destroyed streams - #23785

Closed
mcollina wants to merge 1 commit into
nodejs:masterfrom
mcollina:fix-23730
ConversationCommits (1)ChecksFiles changed
Closed

Ensure async iteration works with destroyed streams#23785
mcollina wants to merge 1 commit into
nodejs:masterfrom
mcollina:fix-23730

Conversation

mcollina commented Oct 20, 2018 •
edited
Loading

Copy link
Copy Markdown
Member

This PR is twofold:

  1. makes sure that .destroy() works correctly on zlib streams (before it was not cleaning up the _handle) done in zlib: do not leak on destroy #23734
  2. fixes support for async-iterating destroyed streams.

Fixes #23730 .

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

nodejs-github-bot commented Oct 20, 2018

Copy link
Copy Markdown
Collaborator

@mcollina build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1317/pipeline

nodejs-github-bot added the zlib Issues and PRs related to the zlib subsystem. label Oct 20, 2018
mcollina mentioned this pull request Oct 20, 2018
Closed

mafintosh commented Oct 20, 2018

Copy link
Copy Markdown
Member

@mcollina does this supersede my zlib destroy leak PR?

Hakerh400 commented Oct 20, 2018

Copy link
Copy Markdown
Contributor

https://github.com/nodejs/node/blob/0983ed818e5b93b6db794b04b58425d8d4ffc43d/lib/internal/streams/async_iterator.js#L90-L91

Nit: destroy(err)

mcollina commented Oct 20, 2018

Copy link
Copy Markdown
Member Author

@mafintosh which one? Very likely that should be merged first.

devsnek approved these changes Oct 20, 2018
View reviewed changes

devsnek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

It seems this relies on a race condition? I won't pretend to know anything about our crazy streams but I want to just verify what's going on here is what is intended.

Comment thread lib/internal/streams/async_iterator.js Outdated

devsnek Oct 20, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

any reason for rejects rather than reject?

mcollina Oct 20, 2018

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

none, I’ll fix it.

mcollina commented Oct 20, 2018

Copy link
Copy Markdown
Member Author

@devsnek this is normalizing a race condition that we cannot get rid of inside our stream machinery :/.

devsnek added stream Issues and PRs related to the stream subsystem. experimental Issues and PRs related to experimental features. labels Oct 20, 2018

mafintosh commented Oct 20, 2018

Copy link
Copy Markdown
Member

@mcollina this one #23734

mcollina commented Oct 20, 2018

Copy link
Copy Markdown
Member Author

Ah I didn’t know we were working on essentially the same issue!

Can you pick up the change that I did inside processCallback? It might even be worth a test on its own. I’ll rebase this on top of yours.

lpinca reviewed Oct 20, 2018
View reviewed changes
Comment thread lib/internal/streams/async_iterator.js Outdated
addaleax reviewed Oct 21, 2018
View reviewed changes
Comment thread lib/zlib.js Outdated

addaleax Oct 21, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can you add a comment here describing what this condition means?

Comment thread lib/zlib.js Outdated

addaleax Oct 21, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This changes timing for the close callback… is that intentional?

addaleax mentioned this pull request Oct 21, 2018
Closed
4 tasks

mcollina commented Oct 23, 2018

Copy link
Copy Markdown
Member Author

CI: https://ci.nodejs.org/job/node-test-pull-request/18090/

PTAL, this should be ready for review.

mcollina commented Oct 24, 2018

Copy link
Copy Markdown
Member Author

CI: https://ci.nodejs.org/job/node-test-pull-request/18100/

mcollina commented Oct 24, 2018

Copy link
Copy Markdown
Member Author

@lpinca @addaleax is this ok to land for you?

mafintosh reviewed Oct 24, 2018
View reviewed changes
Comment thread lib/internal/streams/async_iterator.js Outdated

mafintosh Oct 24, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

could use stream.finished to simplify in the future, but 👍

lpinca approved these changes Oct 24, 2018
View reviewed changes
Comment thread test/parallel/test-stream-readable-async-iterators.js Outdated

lpinca Oct 24, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality
Suggested change
assert.strictEqual(err, e);
assert.strictEqual(e, err);

Comment thread test/parallel/test-stream-readable-async-iterators.js Outdated

lpinca Oct 24, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality
Suggested change
assert.strictEqual(err, e);
assert.strictEqual(e, err);

addaleax removed the zlib Issues and PRs related to the zlib subsystem. label Oct 24, 2018

addaleax commented Oct 24, 2018

Copy link
Copy Markdown
Member

@mcollina There’s nothing speaking against landing this as far as I am concerned :)

mcollina commented Oct 24, 2018

Copy link
Copy Markdown
Member Author

CI: https://ci.nodejs.org/job/node-test-pull-request/18110/

PTAL, I've switched to use finished instead.

lpinca commented Oct 24, 2018

Copy link
Copy Markdown
Member

Still LGTM.

mmarchini added the author ready PRs that have at least one approval, no outstanding review comments, and a CI started. label Oct 24, 2018

mcollina commented Oct 24, 2018

Copy link
Copy Markdown
Member Author

Landed in 3ec8cec

mcollina closed this Oct 24, 2018
mcollina deleted the fix-23730 branch October 24, 2018 13:25
mcollina added a commit that referenced this pull request Oct 24, 2018
stream: async iteration should work with destroyed stream
3ec8cec 
Fixes #23730.

PR-URL: #23785
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>

mafintosh commented Oct 24, 2018

Copy link
Copy Markdown
Member

(Also still LGTM for the record)

targos pushed a commit that referenced this pull request Oct 24, 2018
stream: async iteration should work with destroyed stream
aaddf97 
Fixes #23730.

PR-URL: #23785
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
This was referenced Oct 26, 2018
Closed
Closed
targos mentioned this pull request Oct 27, 2018
Merged
This was referenced Nov 2, 2018
Closed
Closed
Closed
Closed

MylesBorins commented Nov 26, 2018

Copy link
Copy Markdown
Contributor

Landed in 10.x with 398418d and b1e1fe4

Please lmk if it should be backed out

mcollina commented Nov 26, 2018 via email

Copy link
Copy Markdown
Member Author

MylesBorins pushed a commit that referenced this pull request Nov 26, 2018
stream: async iteration should work with destroyed stream
dbd37ab 
Fixes #23730.

PR-URL: #23785
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
codebytere mentioned this pull request Nov 27, 2018
Closed
rvagg pushed a commit that referenced this pull request Nov 28, 2018
stream: async iteration should work with destroyed stream
fca312a 
Fixes #23730.

PR-URL: #23785
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
MylesBorins pushed a commit that referenced this pull request Nov 29, 2018
stream: async iteration should work with destroyed stream
ce79ae6 
Fixes #23730.

PR-URL: #23785
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
codebytere mentioned this pull request Nov 29, 2018
Merged
jaydenseric mentioned this pull request Jan 10, 2019
Closed
rektide mentioned this pull request Mar 6, 2019
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

jasnell jasnell left review comments
addaleax addaleax left review comments
lpinca lpinca approved these changes
+3 more reviewers
mafintosh mafintosh approved these changes
mmarchini mmarchini approved these changes
devsnek devsnek approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author ready PRs that have at least one approval, no outstanding review comments, and a CI started. experimental Issues and PRs related to experimental features. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Stream destruction breaks async-iteration and ends nodejs event loop

10 participants

Back | FazBrowse Home | New Git URL