This pull request will require a lot of discussion. Other puzzles:

• Why do methods have an inconsistent number of redundant and incomplete (only names and optional indicator) signatures? Examples:

  • asyncResource.emitDestroy has 0.
  • asyncResource.asyncId has 1.
  • buf.readDoubleBE has 2.

• Where did all of the constructors go?

$ grep -l '"ctors"' out/doc/api/*.json
out/doc/api/all.json
out/doc/api/crypto.json
out/doc/api/stream.json
$ git grep -l '## Constructor:' doc/api/*.md
doc/api/inspector.md
doc/api/stream.md
doc/api/url.md
doc/api/vm.md
$ git grep -l '## new ' doc/api/*.md
doc/api/assert.md
doc/api/async_hooks.md
doc/api/buffer.md
doc/api/console.md
doc/api/crypto.md
doc/api/errors.md
doc/api/http.md
doc/api/net.md
doc/api/perf_hooks.md
doc/api/stream.md
doc/api/string_decoder.md
doc/api/tls.md
doc/api/util.md
doc/api/v8.md
doc/api/vm.md
doc/api/worker_threads.md

My current code is rejecting the constructor for console as the first signature defines parameters that are included in the list list below (at least not as first level items):

### new Console(stdout[, stderr][, ignoreErrors])
### new Console(options)
<!-- YAML
...
-->

* `options` {Object}
  * `stdout` {stream.Writable}
  * `stderr` {stream.Writable}
  * `ignoreErrors` {boolean} Ignore errors when writing to the underlying
    streams. **Default:** `true`.
  * `colorMode` {boolean|string} Set color support for this `Console` instance.
    Setting to `true` enables coloring while inspecting values, setting to
    `'auto'` will make color support depend on the value of the `isTTY` property
    and the value returned by `getColorDepth()` on the respective stream.
    **Default:** `'auto'`.

I can fix this up, but not seeing the Console constructor in the current output means that I'm flying blind.

Latest issue: files like buffer.json sometimes parses stability correctly, e.g.:

"stability": 2

And other times misses it entirely:

"desc": "<p>Stability: 0 - Deprecated: Use\n..."

The issue here is the embedded newline. The bug here is that stabilityExpr needs dotAll or equivalent.

This pull request is no longer in progress, nor is it ready to land. It depends on #21782 and #21780. In addition to providing fixes to the bugs mentioned above and listed in jsondiff.js, it differs from the output currently produced in that backslashes won't show up in names (for an example, look for "name": "\_\_filename" in modules.json.

Work remaining to be done before this lands: rename json2.js to json.js (overwriting the original source) and deleting jsondiff.js. For the moment, I'm leaving both sources and the diff tool with the thought that it would help reviewers.

Once #21490 lands, further refactoring and cleanup is possible.
html.js and json.js can be changed to export unified plugins and generate.js can implement the main pipeline that produces both. This will make it easy for other tools to be added to the pipeline, for example, see #21723. Also at that time, remark can be removed from package.json and the node_modules directory could be cleaned up.

This pull request is no longer in progress, nor is it ready to land. It depends on #21782 and #21780.

Removed in progress and applied blocked.

Removed the blocked label.

CI: https://ci.nodejs.org/job/node-test-pull-request/15967/

CI is all green.

I've compared the diff in *.html results. It seems we have only this significant difference: some comments disappear in the new HTML docs, such as:

<!--introduced_in=v...-->

<!-- type=misc -->
<!--type=class-->
<!--type=module-->
<!-- type=global -->
<!-- type=var -->
<!--type=event-->
<!--type=example-->

<!-- name=dgram -->
<!--name=esm-->
<!--name=fs-->
<!--name=module-->
<!--name=SIGINT, SIGHUP, etc.-->
<!--name=querystring-->
<!--name=vm-->

Not sure if this is breaking in any aspect. Maybe we can ignore this change as these comments seem needed at the generation stage only.

Ignorable note: json.js uses many iterations over elements using their properties with repetitive retrieving. Maybe it is worth to cache the properties once, especially if the element itself is not used. Say:

      nodes.forEach((node, i) => {

->

      nodes.forEach(({ type, value, children }, i) => {

or:

  return nodes.map((node) => {

->

  return nodes.map(({ type, position, value, children }) => {

etc.
Maybe this will speed up the process a little.

CC @nodejs/documentation @nodejs/build @nodejs/build-files: please, let us know if you plan to review this PR in the near future. Otherwise, I will land it, say, on Wednesday.

CI-lite: https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/339/

@rubys This need rebasing (probably after b38b8d3).

rebased

CI: https://ci.nodejs.org/job/node-test-pull-request/15996/

Landed in f41dd55
Thank you!

Depends on #21616 to land on v10.x-staging