http2: Introducing http/2 by jasnell · Pull Request #14239 · nodejs/node · GitHub

At long last: The initial experimental implementation of HTTP/2.

This is an accumulation of the work that has been done in the nodejs/http2
repository, squashed down to a couple of commits. The original commit
history has been preserved in the nodejs/http2 repository.

Contributors to this work include:

Anna Henningsen / @addaleax
Colin Ihrig / @cjihrig
Daniel Bevenius / @danbev
James M Snell / @jasnell
Jun Mukai / @jmuk
Kelvin Jin / @kjin
Matteo Collina / @mcollina
Robert Kowalski / @robertkowalski
Santiago Gimeno / @santigimeno
Sebastiaan Deckers / @sebdeckers
Yosuke Furukawa / @yosuke-furukawa

This PR introduces the nghttp2 C library as a new dependency. This library
provides the majority of the HTTP/2 protocol implementation, with the rest
of the code here providing the mapping of the library into a usable JS API.

Within src, a handful of new node_http2_.c and node_http2_.h files are
introduced. These provide the internal mechanisms that interface with nghttp
and define the process.binding('http2') interface.

The JS API is defined within internal/http2/*.js.

There are two APIs provided: Core and Compat.

The Core API is HTTP/2 specific and is designed to be as minimal and as
efficient as possible.

The Compat API is intended to be as close to the existing HTTP/1 API as
possible, with some exceptions.

Tests, documentation and initial benchmarks are included.

The http2 module is gated by a new --expose-http2 command line flag.
When used, require('http2') will be exposed to users. Note that there
is an existing http2 module on npm that would be impacted by the introduction
of this module, which is the main reason for gating this behind a flag.

When using require('http2') the first time, a process warning will be
emitted indicating that an experimental feature is being used.

To run the benchmarks, the h2load tool (part of the nghttp project) is
required: ./node benchmarks/http2/simple.js benchmarker=h2load. Only
two benchmarks are currently available.

Additional configuration options to enable verbose debugging are provided:

$ ./configure --debug-http2 --debug-nghttp2
$ NODE_DEBUG=http2 ./node

The --debug-http2 configuration option enables verbose debug statements
from the src/node_http2_* files. The --debug-nghttp2 enables the nghttp
library's own verbose debug output. The NODE_DEBUG=http2 enables JS-level
debug output.

The following illustrates as simple HTTP/2 server and client interaction:

(The HTTP/2 client and server support both plain text and TLS connections)

const req = client.request({ ':path': '/some/path' });
req.on('data', (chunk) => { /* do something with the data */ });
req.on('end', () => {
  client.destroy();
});

// Plain text (non-TLS server)
const server = http2.createServer();
server.on('stream', (stream, requestHeaders) => {
  stream.respond({ ':status': 200 });
  stream.write('hello ');
  stream.end('world');
});
server.listen(80);

const http2 = require('http2');
const client = http2.connect('http://localhost');

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

http2

[Edit:ofrobots: fixed github handle for Kelvin Jin]

Just a thought: separating the new dep checkin to a separate commit might make review easier?

This thing is going to be massive and difficult to review no matter how it is split up, unfortunately.

Nitpick: typo on this comment

That's in the upstream dependency so it would need to be fixed there :-) https://github.com/nghttp2/nghttp2

Fixed upstream nghttp2/nghttp2#958

A nit: redundant after #14021

thinking about it, I think I need to go add the crypto skip in a couple of the other tests.

Came up in downstream code review by @lucaslago nodejs/node#14239 (comment)

@jasnell We also split out the deps commits for node-inspect and for the inspector iirc; that worked out pretty well and did help with reviewing. :)

Also, for N-API, async_hooks, and some other PRs, we used Author: name <email> metadata (example: 56e881d) for specifying multiple authors. Feel invited to adapt that format. :) (That has the nice side effect of not pinging everybody everytime you rebase/the commit is landed/etc.)

This does the same thing as stream.priority(options), right? In that case I think I’d prefer not to expose it (same thing for rstStream)

Currently, yes, but eventually this will support the ability to pass in a numeric stream identifier rather than the Http2Stream object. The HTTP/2 spec allows priorities and rst_stream frames to be sent independently, even without a stream having been previously established.

The colon prefix in :status looks a bit odd to me. I'm sure there have been careful considerations made as to why that makes sense. Would you mind sharing the reasoning behind using colon prefixes? Thanks!

@yoshuawuyts it's part of the HTTP2 spec: https://http2.github.io/http2-spec/#HttpResponse

@mcollina cool, thanks for answering!

…

On Sat, Jul 15, 2017, 14:47 Matteo Collina ***@***.***> wrote: @yoshuawuyts <https://github.com/yoshuawuyts> it's part of the HTTP2 spec: https://http2.github.io/http2-spec/#HttpResponse — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#14239 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACWleuhN8XM3Dk7G-OD3ZsFjZciMOVUsks5sOLThgaJpZM4OYrK1> .

Awesome work! Above you said the following:

The Compat API is intended to be as close to the existing HTTP/1 API as possible, with some exceptions.

I tried looking in the documentation included in this PR, but couldn't figure out what these exceptions are. Can you ellaborate? Is the compatibility API just the same how the "spdy" module does the HTTP/2 -> 1 compatibility API or different somehow? I'm not really looking for a comparison to the "spdy" module, but thought if you knew that would help, otherwise would love to see the docs around what these differences are.

@dougwilson .... I'm hoping to get that compat api documented this next week... Including those caveats.

http2 should also be added to all.md.

`RST_STREAM`?

nghttp2.org

Thanks @jasnell ! Please feel free to ping me when you do, I would love to take a look.

missing ]

According to using in text,

[ServerHttp2Stream]: [ClientHttp2Stream]: [Http2Stream]:

should be:

[`ServerHttp2Stream`]: [`ClientHttp2Stream`]: [`Http2Stream`]:

Missing reference for [Buffer][]

Missing ]

Issue with the first examples from the main doc: #14304

@dougwilson Some notable differences, off the top of my head:

Most private variables are not supported. Only the documented methods, properties, and events are supported. Code that relies on undocumented behaviour is not supported by the compat api. E.g. no more _header checking; use headersSent instead.
The request and response objects do not inherit from http.IncomingMessage and http.ServerResponse respectively. This poses a challenge for express@4, in my experience.
No status message in sendHead because HTTP/2 dropped the concept of text status messages. Status code is now the :status header. A process warning is emitted if the (optional) status message is passed. The message itself is ignored. Status code and rest of the headers are sent.
The socket property should not be written to. HTTP/2 has a framing layer and does not directly expose the socket. Only the low-level nghttp2 library deals with the socket. However meta properties on the socket can be useful, like detecting http vs https, address/port, TLS info, etc.
Only status codes 1xx-5xx are supported. This follows the HTTP spec. Previously in the HTTP/1 API status codes 1xx-9xx were allowed.
Extensions:
- The [req|res].stream property exposes the HTTP/2 stream, which in turn exposes the HTTP/2 session.
- res.createPushResponse(headers, cb) can use used to send a PUSH_PROMISE frame. The callback receives the corresponding Http2ServerResponse instance via which headers and a body can be sent.

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

At long last: The initial *experimental* implementation of HTTP/2. This is an accumulation of the work that has been done in the nodejs/http2 repository, squashed down to a couple of commits. The original commit history has been preserved in the nodejs/http2 repository. This PR introduces the nghttp2 C library as a new dependency. This library provides the majority of the HTTP/2 protocol implementation, with the rest of the code here providing the mapping of the library into a usable JS API. Within src, a handful of new node_http2_*.c and node_http2_*.h files are introduced. These provide the internal mechanisms that interface with nghttp and define the `process.binding('http2')` interface. The JS API is defined within `internal/http2/*.js`. There are two APIs provided: Core and Compat. The Core API is HTTP/2 specific and is designed to be as minimal and as efficient as possible. The Compat API is intended to be as close to the existing HTTP/1 API as possible, with some exceptions. Tests, documentation and initial benchmarks are included. The `http2` module is gated by a new `--expose-http2` command line flag. When used, `require('http2')` will be exposed to users. Note that there is an existing `http2` module on npm that would be impacted by the introduction of this module, which is the main reason for gating this behind a flag. When using `require('http2')` the first time, a process warning will be emitted indicating that an experimental feature is being used. To run the benchmarks, the `h2load` tool (part of the nghttp project) is required: `./node benchmarks/http2/simple.js benchmarker=h2load`. Only two benchmarks are currently available. Additional configuration options to enable verbose debugging are provided: ``` $ ./configure --debug-http2 --debug-nghttp2 $ NODE_DEBUG=http2 ./node ``` The `--debug-http2` configuration option enables verbose debug statements from the `src/node_http2_*` files. The `--debug-nghttp2` enables the nghttp library's own verbose debug output. The `NODE_DEBUG=http2` enables JS-level debug output. The following illustrates as simple HTTP/2 server and client interaction: (The HTTP/2 client and server support both plain text and TLS connections) ```jt client = http2.connect('http://localhost:80'); const req = client.request({ ':path': '/some/path' }); req.on('data', (chunk) => { /* do something with the data */ }); req.on('end', () => { client.destroy(); }); // Plain text (non-TLS server) const server = http2.createServer(); server.on('stream', (stream, requestHeaders) => { stream.respond({ ':status': 200 }); stream.write('hello '); stream.end('world'); }); server.listen(80); ``` ```js const http2 = require('http2'); const client = http2.connect('http://localhost'); ``` Author: Anna Henningsen <anna@addaleax.net> Author: Colin Ihrig <cjihrig@gmail.com> Author: Daniel Bevenius <daniel.bevenius@gmail.com> Author: James M Snell <jasnell@gmail.com> Author: Jun Mukai Author: Kelvin Jin Author: Matteo Collina <matteo.collina@gmail.com> Author: Robert Kowalski <rok@kowalski.gd> Author: Santiago Gimeno <santiago.gimeno@gmail.com> Author: Sebastiaan Deckers <sebdeckers83@gmail.com> Author: Yosuke Furukawa <yosuke.furukawa@gmail.com> PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Race condition in the closing of the stream causing failure on some platforms. PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Fixes: nodejs/http2#184 Refines the `'socketError'` event a bit and adds a test for the emission of the `'socketError'` event on the server. Client side is tested separately PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Fixes: nodejs/http2#179 Was fixing issue nodejs#179 and encountered a segault that was happening somewhat randomly on session destruction. Both should be fixed PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

* respondWithFD now supports optional statCheck * respondWithFD and respondWithFile both support offset/length for range requests * Fix linting nits following most recent update PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Rather than using the `'fetchTrailers'` event to collect trailers, a new `getTrailers` callback option is supported. If not set, the internals will skip calling out for trailers at all. Expands the test to make sure trailers work from the client side also. PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

@jasnell https://nodejs.org/api/http2.html has many not rendered links now (just search ][ in a browser, skipping headings).

AFAICT the md has has broken links (or rather links with no definition in the footer) in certain areas. Some were fixed in 85d7d97

Cross ref: #14857

Setting don't land on v6.x. Assuming we are not going to backport this

jasnell added the semver-minor PRs that contain new features and should be released in the next minor version. label Jul 14, 2017

nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Jul 14, 2017

jasnell added experimental Issues and PRs related to experimental features. http2 Issues or PRs related to the http2 subsystem. labels Jul 14, 2017

jasnell changed the title Initial pr http2: Introducing http/2 Jul 14, 2017

lucaslago reviewed Jul 14, 2017

View reviewed changes

vsemozhetbyt reviewed Jul 14, 2017

View reviewed changes

sebdeckers added a commit to sebdeckers/nghttp2 that referenced this pull request Jul 14, 2017

fix: typo

c601e60

Came up in downstream code review by @lucaslago nodejs/node#14239 (comment)

sebdeckers mentioned this pull request Jul 14, 2017

fix: typo nghttp2/nghttp2#958

Merged

addaleax reviewed Jul 15, 2017

View reviewed changes

vsemozhetbyt reviewed Jul 15, 2017

View reviewed changes

vsemozhetbyt reviewed Jul 16, 2017

View reviewed changes

vsemozhetbyt mentioned this pull request Jul 16, 2017

http2: code examples does not work on Windows with port 80 #14304

Closed

jasnell added a commit to jasnell/node that referenced this pull request Aug 13, 2017

deps: add nghttp2 dependency

a22543c

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

jasnell added a commit to jasnell/node that referenced this pull request Aug 13, 2017

http2: add tests and benchmarks

837dfe4

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

jasnell added a commit to jasnell/node that referenced this pull request Aug 13, 2017

http2: fix documentation nits

8357bd0

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

jasnell added a commit to jasnell/node that referenced this pull request Aug 13, 2017

doc: include http2.md in all.md

295e4b1

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

jasnell added a commit to jasnell/node that referenced this pull request Aug 13, 2017

http2: minor cleanup

a2045e9

PR-URL: nodejs#14239 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

This was referenced Aug 14, 2017

[8.x] Backport multiple http2 PRs #14813

Closed

v8.4.0 proposal #14811

Merged

mcollina mentioned this pull request Sep 28, 2017

Revisit putting new features behind flags nodejs/TSC#369

Closed

Hackzzila mentioned this pull request May 2, 2018

brotli: add brotli support #20458

Closed

4 tasks

sam-github mentioned this pull request Feb 7, 2019

tls: renegotiate should take care of its own state #25997

Closed

3 tasks

mildsunrise mentioned this pull request Oct 6, 2019

HTTP/2 Host header disallowed #29858

Closed

LiviaMedeiros mentioned this pull request May 9, 2025

test: fix executable flags #58250

Merged

http2: Introducing http/2 by jasnell · Pull Request #14239 · nodejs/node · GitHub

Uh oh!

Uh oh!

http2: Introducing http/2 - #14239

http2: Introducing http/2#14239jasnell wants to merge 26 commits intonodejs:masternodejs/node:masterfrom jasnell:initial-prjasnell/node:initial-prCopy head branch name to clipboard

Conversation

jasnell commented Jul 14, 2017 • edited by ofrobots Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Affected core subsystem(s)

Uh oh!

Fishrock123 commented Jul 14, 2017

Uh oh!

jasnell commented Jul 14, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vsemozhetbyt Jul 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

addaleax commented Jul 15, 2017

Uh oh!

addaleax commented Jul 15, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yoshuawuyts commented Jul 15, 2017

Uh oh!

mcollina commented Jul 15, 2017

Uh oh!

yoshuawuyts commented Jul 15, 2017 via email

Uh oh!

dougwilson commented Jul 15, 2017

Uh oh!

jasnell commented Jul 15, 2017

Uh oh!

vsemozhetbyt commented Jul 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dougwilson commented Jul 15, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vsemozhetbyt commented Jul 16, 2017

Uh oh!

sebdeckers commented Jul 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vsemozhetbyt commented Aug 16, 2017

Uh oh!

refack commented Aug 16, 2017

Uh oh!

refack commented Aug 16, 2017

Uh oh!

MylesBorins commented Jan 11, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

http2: Introducing http/2#14239
jasnell wants to merge 26 commits into
nodejs:masterfrom
jasnell:initial-pr

jasnell commented Jul 14, 2017 •
edited by ofrobots
Loading

vsemozhetbyt Jul 14, 2017 •
edited
Loading

vsemozhetbyt commented Jul 15, 2017 •
edited
Loading

sebdeckers commented Jul 16, 2017 •
edited
Loading