FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

stream: 'readable' have precedence over flowing by mcollina · Pull Request #18994 · nodejs/node · GitHub

/ node Public

stream: 'readable' have precedence over flowing - #18994

Closed
mcollina wants to merge 1 commit into
nodejs:masterfrom
mcollina:readable-and-data
Closed

stream: 'readable' have precedence over flowing#18994
mcollina wants to merge 1 commit into
nodejs:masterfrom
mcollina:readable-and-data

Conversation

Copy link
Copy Markdown
Member

In Streams3 the 'readable' event/.read() method had a lower precedence
than the 'data' event that made them impossible to use them together.
This make .resume() a no-op if there is a listener for the
'readable' event, making the stream non-flowing if there is a
'data'  listener.

Fixes: #18058

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

stream, http

nodejs-github-bot added the stream Issues and PRs related to the stream subsystem. label Feb 26, 2018
mcollina added semver-major PRs that contain breaking changes and should be released in the next major version. dont-land-on-v4.x labels Feb 26, 2018

mcollina commented Feb 26, 2018
edited
Loading

Copy link
Copy Markdown
Member Author

mscdex commented Feb 26, 2018

Copy link
Copy Markdown
Contributor

Wouldn't removeAllListeners() to be handled as well, since there could be no 'removeListener' event handlers?

Copy link
Copy Markdown
Member Author

@mscdex you are right. I didn't account for https://github.com/nodejs/node/blob/master/lib/events.js#L324-L336. Do you think I can just listen for 'removeListener' instead? I thought this approach could theoretically be more performing, but it's an edge case anyway.

Copy link
Copy Markdown
Member

@mcollina so basically just suspends flowing when readable is set?

Copy link
Copy Markdown
Member Author

@mafintosh that's the goal yes. It also restores the behavior after 'readable' is removed.
I'm not sure this is something we want to land in this form (or we might want to refactor).
This is the fix for #18058.

Copy link
Copy Markdown
Member

@mcollina wondering if we should even fix this. the same behaivor described in the issue would happen if there are two consumers of the readable event anyway (which is what happens in flowing mode). not too much of a fan of implicit event listener side effects

Copy link
Copy Markdown
Member Author

I am good with not fixing it. But it’s a discussion to have.

This fixed a major usability problem if you want to have both a on(‘data’) and on(‘readable’)/read() at the same time (with pipe) read() will always return null.

I’m ok if we want to remove the resume side effect when removing ‘readable’, as that might be confusing. However we should really be updating readableListening: this is also a bug without the fix for read() itself.

Copy link
Copy Markdown
Member

Removing the removeListener stuff but keeping the other part sounds like a good middleway to me 👍

BridgeAR commented Mar 6, 2018

Copy link
Copy Markdown
Member

@nodejs/streams @mafintosh PTAL

BridgeAR commented Mar 6, 2018

Copy link
Copy Markdown
Member

mcollina commented Mar 7, 2018

Copy link
Copy Markdown
Member Author

@nodejs/tsc what do you think?

An alternative approach might be to remove the readableListening variable and just use listenerCount, hopefully it would be fast enough.

Comment thread doc/api/stream.md Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

A nit: missing period.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

did you mean to add these logs?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Yes, they follow the prevailing pattern in this test file. Without the console.log statements, this is undebuggable.

Copy link
Copy Markdown
Member

Lots of good tests! LGTM from me 👍

mcollina force-pushed the readable-and-data branch from 03e66ed to 44fad80 Compare March 14, 2018 21:17

Copy link
Copy Markdown
Member Author

mcollina requested review from a team and lpinca March 14, 2018 21:28

Copy link
Copy Markdown
Member Author

Tagging @nodejs/tsc because it is semver-major.

mcollina added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Apr 4, 2018
Comment thread doc/api/stream.md Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Nit: not sure why this was changed and it doesn't really matter but for consistency I would keep the arrow function.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@lpinca it is using this.read() below

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Oops, ignore me.

mcollina force-pushed the readable-and-data branch from 44fad80 to 8085e2e Compare April 4, 2018 15:58
mcollina added this to the 10.0.0 milestone Apr 4, 2018

mcollina commented Apr 4, 2018

Copy link
Copy Markdown
Member Author

Trott removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Apr 4, 2018
Comment thread doc/api/stream.md Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can you avoid the line break and capitalize the sentence?

Comment thread doc/api/stream.md Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

ditto

Comment thread lib/_stream_readable.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Might be good to have chunk printed out here as well

Comment thread lib/_stream_readable.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can you change to comment above this line so that it says why we do this, rather than what we do? I assume it’s about the this.resume() call below?

In Streams3 the 'readable' event/.read() method had a lower precedence
than the `'data'` event that made them impossible to use them together.
This make `.resume()` a no-op if there is a listener for the
`'readable'` event, making the stream non-flowing if there is a
`'data'`  listener.

Fixes: nodejs#18058
mcollina force-pushed the readable-and-data branch from 8085e2e to 578a7b9 Compare April 4, 2018 21:11

mcollina commented Apr 4, 2018

Copy link
Copy Markdown
Member Author

@addaleax PTAL

mcollina commented Apr 5, 2018

Copy link
Copy Markdown
Member Author

mcollina commented Apr 6, 2018

Copy link
Copy Markdown
Member Author

Landed as cf5f986

mcollina closed this Apr 6, 2018
mcollina deleted the readable-and-data branch April 6, 2018 12:17
mcollina added a commit that referenced this pull request Apr 6, 2018
In Streams3 the 'readable' event/.read() method had a lower precedence
than the `'data'` event that made them impossible to use them together.
This make `.resume()` a no-op if there is a listener for the
`'readable'` event, making the stream non-flowing if there is a
`'data'`  listener.

Fixes: #18058

PR-URL: #18994
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>

Copy link
Copy Markdown
Contributor

Seems like failures in cf5f986#diff-573b8412079b987e160a3fde511d5b9b are showing up in a couple places

In a couple CI runs in #19201 and also in http://github.com/nodejs/node/pull/19924

@mcollina thoughts?

not ok 750 parallel/test-http-readable-data-event
  ---
  duration_ms: 0.725
  severity: fail
  stack: |-
    assert.js:79
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: 'Hello World!Hello again later!' strictEqual 'Hello World!'
        at IncomingMessage.res.on.common.mustCall (/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/parallel/test-http-readable-data-event.js:43:14)
        at IncomingMessage.<anonymous> (/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/common/index.js:467:15)
        at IncomingMessage.emit (events.js:182:13)
        at IncomingMessage.Readable.read (_stream_readable.js:489:10)
        at IncomingMessage.res.on.common.mustCall (/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/parallel/test-http-readable-data-event.js:36:20)
        at IncomingMessage.<anonymous> (/home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/test/common/index.js:467:15)
        at IncomingMessage.emit (events.js:182:13)
        at emitReadable_ (_stream_readable.js:537:12)
        at process._tickCallback (internal/process/next_tick.js:174:19)
  ...

Copy link
Copy Markdown
Member Author

@MylesBorins We are already tracking it in #19905.

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-major PRs that contain breaking changes and should be released in the next major version. stream Issues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.


Back | FazBrowse Home | New Git URL