quic: apply multiple quic performance and packet handling improvements - #63267
Closed
jasnell wants to merge 28 commits into
Closed
quic: apply multiple quic performance and packet handling improvements#63267jasnell wants to merge 28 commits into
jasnell wants to merge 28 commits into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
Copy link
Copy Markdown
Collaborator
|
Review requested:
|
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
jasnell
force-pushed
the
jasnell/quic-backend-improvements
branch
from
e921c66 to
ef9e8da
Compare
May 13, 2026 03:54
This comment was marked as outdated.
This comment was marked as outdated.
jasnell
force-pushed
the
jasnell/quic-backend-improvements
branch
4 times, most recently
from
bbe613a to
6ef4e68
Compare
May 15, 2026 13:41
Copy link
Copy Markdown
Contributor
|
This diff is quite large. Can you provide a short summary of the most relevant parts? Such as maybe notable performance improvement areas, the bug fix, and whatever else seems relevant. That'll help expedite reviews, thanks! |
Sorry, something went wrong.
jasnell
force-pushed
the
jasnell/quic-backend-improvements
branch
2 times, most recently
from
bfc3adc to
c9c714e
Compare
May 15, 2026 16:56
Copy link
Copy Markdown
Member
Author
|
Just an idea of where we're at... running a local 'hello world' benchmark using h2load as the client driver on the same 88 core machine... 4 runs with 100,000 requests over 100 concurrent connections: 
The rps is not as high as I'd like and http/1 still beats it by a long way but it's a solid result for current state. The primary performance bottleneck right now is the JS side, GC, and the C++/JS boundary cross. We lose a fair amount of throughput in the context switching. My current goal is to get the per-request down to 10 microseconds. |
Sorry, something went wrong.
Copy link
Copy Markdown
Member
Author
|
Ok, with a bit of perf tunning and mangling with both h2load options and v8 perf tuning options on the server, with a node binary built with pointer compression enabled... we can get ~101k-102k rps. Still slower than http/1, on par with http/2 when pointer compression is enabled. Overall a great result given that I've only just started the perf tuning of the implementation. /tmp/opencode/local/bin/h2load --h3 -n 200000 -c 5 -w 26 -W 30 -m 10 -D 2 https://127.0.0.1:4433
starting benchmark...
spawning thread #0: 5 total client(s). Timing-based test with 0s of warm-up time and 2s of main duration for measurements.
Warm-up started for thread #0.
progress: 20% of clients started
progress: 40% of clients started
progress: 60% of clients started
progress: 80% of clients started
progress: 100% of clients started
Warm-up phase is over for thread #0.
Main benchmark duration is started for thread #0.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Certificate: ECDSA P-256 256 bits
Negotiated Group: X25519
Resumption: no
Application protocol: h3
Main benchmark duration is over for thread #0. Stopping all clients.
Stopped all clients for thread #0
finished in 2.02s, 102130.00 req/s, 2.23MB/s
requests: 204260 total, 204310 started, 204260 done, 204260 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 204260 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4.46MB (4676960) total, 797.97KB (817120) headers (space savings 87.50%), 2.14MB (2247080) data
UDP datagram: 33273 sent, 20462 received
min max median p95 p99 mean sd +/- sd
request : 150us 53.98ms 435us 490us 813us 487us 1.28ms 99.59%
connect : 4.89ms 5.64ms 5.30ms 5.64ms 5.64ms 5.28ms 295us 60.00%
TTFB : 8.11ms 8.25ms 8.18ms 8.25ms 8.25ms 8.18ms 54us 60.00%
req/s : 20402.15 20442.34 20417.92 20442.34 20442.34 20421.97 17.40 60.00%
min RTT : 22us 115us 38us 115us 115us 57us 40us 80.00%
smoothed RTT: 86us 226us 164us 226us 226us 163us 60us 60.00%
packets sent: 6219 7436 6373 7436 7436 6656.60 545.25 80.00%
packets recv: 4092 4099 4092 4099 4099 4094.40 3.36 80.00%
packets lost: 0 0 0 0 0 0.00 0.00 100.00%
GRO packets : 1 1 1 1 1 1.00 0.00 100.00%
|
Sorry, something went wrong.
Signed-off-by: James M Snell <jasnell@gmail.com>
Use a uv_check_t on BindingData to process outbound pending packet send, and use TrySend for actually sending packets when possible. Results in an 8% improvement in req/s and ~24% improvement in p95 latency. Also sets us up better for future improvements in libuv if the changes proposed in libuv/libuv#5116 are accepted. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6
Signed-off-by: James M Snell <jasnell@gmail.com>
Improves overall performance and sets us up for eventual support for GRO/GSO batching in libuv in the future. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6
Set up for when libuv eventually supports ECN marking. Pass the ECN marking stuff into ngtcp2. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Use a uv_check_t on BindingData to process outbound pending packet send, and use TrySend for actually sending packets when possible. Results in an 8% improvement in req/s and ~24% improvement in p95 latency. Also sets us up better for future improvements in libuv if the changes proposed in libuv/libuv#5116 are accepted. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Improves overall performance and sets us up for eventual support for GRO/GSO batching in libuv in the future. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Set up for when libuv eventually supports ECN marking. Pass the ECN marking stuff into ngtcp2. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Every stream, session, and endpoint creates aliased structs for stats and state. These were creating v8::ArrayBuffer allocations and views for each instance, which is expensive. This adds a new arena mechanism for AliasedStructs that allocates in pages and allows Streams and Sessions to share the same underlying ArrayBuffer for their stats and state. Since these are never exposed to users, this is safe and results in a significant reduction in allocation counts. Each arena maintains a freelist of pages, where each page is a max of 16KB bytes. Pages are lazily allocated and freed as needed. Each slot in the arena corresponds to a single struct instance, and the slot index is used to calculate the byte offset within the page for that struct's view. The perf improvement is modest but measurable. The key benefit is in reduced memory fragmentation and GC overhead. Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
* Adds invalid this protections in the stats classes * Collects internal private fields into a single private symbol to reduce the number of private field accesses * Adds and corrects isQuic* class checks * Preserve stream id and direction after destroy * Improve stats close snapshotting * Fixes a handful of other bugs and doc issues Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95
pushed a commit
that referenced
this pull request
May 22, 2026
Signed-off-by: James M Snell <jasnell@gmail.com> Assisted-by: Opencode:Opus 4.6 PR-URL: #63267 Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Copy link
Copy Markdown
Contributor
|
@jasnell: Can I ask something about this PR and the performance optimization? I do not know if it is the parser of nghttp3 that changes the calling pattern, or it is something else. |
Sorry, something went wrong.
Copy link
Copy Markdown
Contributor
|
I found the underlying cause. I have drafted an issue #64035 |
Sorry, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A number of planned backend changes and improvements in the packet handling. Yields a net performance improvement of around 10% higher rps. Sets us up to be able to better leverage libuv/libuv#5116 if/when that lands (@santigimeno @saghul fyi) which will provide an even larger perf boost. Fixes a bug while we're at it.
@nodejs/quic