| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
@Fishrock123 Good idea converting Writable.prototype.flushSync to be private to lib/internal/process.js. stream._handle.flushSync() is still public accessible though - it only flushes the libuv write queue, not the higher level node stream chunk write queue (a.k.a. bufferedRequest). Which should be fine if documented. |
Sorry, something went wrong.
|
hmm. I am not the biggest fan of changing process.exit() behaviour entirely without having an alternative that actually calls exit(3) as it does now - even though I don't like that either, but the above feels like obstructing future improvements. Please let's not bake it in. Also, would it not be possible and maybe better to do it in cpp, since (1) the whole class could't benefit from it and (2) one would maybe be able to do this without constructing a new buffer. |
Sorry, something went wrong.
|
@eljefedelrodeodeljefe The process.exit() stdio flushing problem can only be fixed at the libuv level. There's no other way. Node programmers have come to expect process.exit() flushes stdout/stderr - even though it was not strictly true historically on all platforms. Having another function like process._exit() that exits immediately without flushing would probably be useful. |
Sorry, something went wrong.
|
yes that is my point. In my try for a patch I was moving process.reallyExit() to node_os.cc naming it os.exit() and aliasing it with reallyExit for backwards compat. There is a valid point for os.exit in case of security issues and similar. Still, voving the flushSync function definition to cpp would be better style, imo. |
Sorry, something went wrong.
I think it might be helpful here (at least to me) if you could explain what kind of improvements you are referring to and why this PR specifically “feels like obstructing future improvements”. |
Sorry, something went wrong.
Why? That is potentially clumsy (have you tried writing JS-esq stuff in V8 c++?) and not any more efficient.
If you mean "if this lands we may not feel like adding a 'graceful-close'", my answer is: ¯_(ツ)_/¯ this has to land regardless. I'll bring it up at the CTC to see if anyone has critical objections to doing stuff before exit, but I think we are fine so long as we shutdown without calling additional user code after the tick. Also consider there is a difference between this and, say, http. One has arbitrary third-party networked connections. The other goes to only one spot per stream. The same could be argued for child process (the connection is perhaps singular and not arbitrary), but since that is not a bug I am not going to include it here. |
Sorry, something went wrong.
|
@kzc This appears to pass on windows; is that that just the result of this never having been an issue on windows? |
Sorry, something went wrong.
|
Sorry, something went wrong.
I can't say as I don't run Windows. I've asked the same question on these tickets and haven't had a definitive response as to whether stdout and stderr block on Windows in node. If they do block, then great - uv_flush_sync would not need to be implemented on Windows and the patch is pretty much good to go. Regarding the timeouts for test-stdout-buffer-flush-on-exit.js, it's to be expected that flushing tens of megabytes (or whatever the test calls for) will take some time. It would take an equal amount of time if the program had not called process.exit() and just exited gracefully. Not sure why freebsd is failing. Unrelated note: The test test-stdout-buffer-flush-on-exit.js only exercises flushing the low level libuv write queue, not the higher level node chunk write queue. The reason for this is that only a single write is done in the test. To test both the libuv write queue and the node chunk write queue you have to do something like this: // this program populates the libuv write queue upon first write over 64K
// then will populate the node stream chunk queue for subsequent writes.
for (var i = 1; i <= 1000; ++i) {
process.stdout.write((i +
': The quick brown fox jumps over the lazy dog.\n').repeat(1500));
}
process.exit(1);
|
Sorry, something went wrong.
|
To possibly fix the test timeout on slower platforms consider changing: [22, 21, 20, 19, 18, 17, 16, 16, 17, 18, 19, 20, 21, 22].forEach((exponent) => {
to something like: [21, 22].forEach((exponent) => {
|
Sorry, something went wrong.
|
@Fishrock123 sigh cc @addaleax see...
|
Sorry, something went wrong.
This is a regression from v1.0.2 (And now v6 for TTYs) and I am going to fix it. Period. I am going to fix it whether or not you disagree that it is a bug.
Whatever the definition of "wrongly" is, it is no longer relevant. We have regressed in a way that breaks a significant amount of programs regardless.
What is a "soft" process.exit()? You mean because of beforeExit()? Not in scope of fixing a regression. For reference's here is man 3 exit:
|
Sorry, something went wrong.
|
Good discussion -.- |
Sorry, something went wrong.
|
I am not going to interfere any further then. |
Sorry, something went wrong.
@Fishrock123 ... Just pointing out, this is not a very "consensus seeking" attitude. Obviously feel free to continue working on it, but the tone of your response here is bordering on being downright rude and inappropriately dismissive. If you disagree with someone, there are more constructive and far more polite ways to say so. The point that I believe @eljefedelrodeodeljefe is making (and I happen to agree with) is that there ought to be an option for exiting without flushing the stdio buffer. Regardless of whether or not you feel flushing stdio by default is appropriate (it is), I consider having the ability to exit immediately without flushing (how process.exit() currently does) to be a perfectly valid use case. |
Sorry, something went wrong.
That's cool, but even exit(3) flushes (some stuff?) so that's not even possible. |
Sorry, something went wrong.
|
Maybe some code helps. Until when do you wanna push this forward? I am gonna make a counter proposal, which even isn't that far off. Just don't have the time in next couple of hours. Not in this case, it doesn't flush our streams. Only what stdout already has. Also it doesn't unwind cpp functions. |
Sorry, something went wrong.
|
Also note: this doesn't change process.abort(). If you absolutely must exit instantly that is your best option even today. |
Sorry, something went wrong.
|
I believe I was quite clearly referring to the current behavior of not flushing the stdio buffers in node/libuv, as process.exit() currently does. I was not referring to the behavior of exit(3), and process.abort() is not what I am referring to, so allow me to restate if I was not clear: I believe the current behavior of process.exit() to exit as soon as possible without flushing the node/libuv buffers for stdout and stderr as proposed by this PR is a valid use case and while the default behavior of process.exit() should be to flush those buffers on exit, there ought to be an option for maintaining the current behavior of process.exit(). |
Sorry, something went wrong.
I can't see a reason why it would be, how? When? Also, this is still a regression so if you want that it needs to be added a separate feature.. Aand if you happen to run on a system that is blocking (e.g. windows as far as I can tell.) you don't even have a choice.. |
Sorry, something went wrong.
|
@Fishrock123 Regarding the test test-stdout-buffer-flush-on-exit.js > 'foo bar baz quux quuz aaa bbb ccc'.repeat(Math.pow(2, 22)).length 138412032 138MB just for one step of the test is too much data. It could be reduced by a factor of 100 considering that node 4.x, 5.x and 6.x only presently outputs around 64KB of data on Linux upon process.exit() |
Sorry, something went wrong.
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See nodejs#6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at nodejs#6456 (comment) Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816 PR-URL: nodejs#6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
Many thanks to thefourtheye and addaleax who helped make the python bits of this possible. See nodejs#6980 for more info regarding the related TTY issues. Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816 PR-URL: nodejs#6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
Closing, I guess this isn't the correct patch and there is still a meta issue. |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
WIP Do not Merge
Checklist
Proposed fix for #6456
Adapted from https://github.com/kzc/node/commit/29997921800e00a22d9f92d24704a0021be03bbf without exposing a new streams API.
Done on a public branch in case someone else needs to take it over.
Initial CI: https://ci.nodejs.org/job/node-test-pull-request/2650/