FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

doc: add guide on maintaining the build files by joyeecheung · Pull Request #16975 · nodejs/node · GitHub

doc: add guide on maintaining the build files by joyeecheung · Pull Request #16975 · nodejs/node · GitHub
Skip to content

Navigation Menu

Sign in

doc: add guide on maintaining the build files - #16975

Closed
joyeecheung wants to merge 2 commits into
nodejs:masterfrom
joyeecheung:doc-makefile
Closed

doc: add guide on maintaining the build files#16975
joyeecheung wants to merge 2 commits into
nodejs:masterfrom
joyeecheung:doc-makefile

Conversation

Copy link
Copy Markdown
Member

This documents how to maintain the build files and some commonly-used targets in the Makefile. It also documents about how to update certain targets to reflect the changes on CI (especially the Raspberry Pis and Linux with FIPS).

This is still a WIP, and only talks about the makefile, configure script and the vcbuild.bat script. There are other build files (e.g. the gyp files) that need to be documented later.

Personally I prefer to leave most of the actual documentation in the code, whiling leave some pointers and necessary explanations in the guide so it's easier to get started with maintaining them.

cc @nodejs/build

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

doc, build

nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Nov 12, 2017

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I am not sure if pasting the help text here is useful..although it does give you an idea about what it does if you don't know what it is.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think the above paragraph is enough. Including the text here will only drift from the actual help text over time.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed with @richardlau , don't think we should duplicate the ./configure --help output.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

We should document the similarity/differences between vcbuild.bat and Makefile, though not necessarily in this PR (I am not familiar enough with vcbuild.bat to be comfortable to write a guide on it anyway)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

that'd be quite a job for someone! one of it's biggest problems is that it lacks most of the smarts that make brings to the table and we only do a partial job of emulating the bits that are needed

joyeecheung added the doc Issues and PRs related to the documentations. label Nov 12, 2017
Comment thread Makefile Outdated

joyeecheung Nov 12, 2017
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Uh, wait, I don't think this is true..? From what I can tell node-test-binary-arm only runs test-ci-js, But then I am not sure who uses this target (test-ci-native) and how are the addons tested on the pis @nodejs/build

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

node-test-binary-arm:

case $RUN_SUBSET in
  addons)
    PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE make test-ci-native
    ;;
  *)
    export CC=should-not-compile CXX=should-not-compile # Prevent building
    PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE TEST_CI_ARGS="--run=${RUN_SUBSET},7" make test-ci-js
    ;;
esac

i.e. it's used for the addons tests on arm-fanned.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Parallel tests are run in parallel by default (by virtue of -J specified to the test runner in Makefile), even if the make command is -j1.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

There's an awkward catch to this that's not documented anywhere that probably should be since I'm finding Jenkins jobs that don't do this properly: a lot of the -ci targets use this PARALLEL_ARGS variable to insert -j X for tools/test.py (-J isn't used in any of the -ci targets). Unfortunately this variable is only set if $JOBS exists, and this isn't automatically set just because you supply -j X to make. In fact, it's kind of difficult to determine the -j argument inside a Makefile in a cross-platform way (but we should probably try, there's an interesting stackoverflow suggestion here). So you have to give JOBS=X when running the -ci targets.

-J is done automatically for tests on non-ci targets so it takes up "as many cores as you have" for tests as @TimothyGu said.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@TimothyGu @rvagg Thanks for the correction, I'll add a note in the Makefile about this, and change this part to just "number of threads used to build the binary"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Typo maintained.

Maybe reword this, e.g. and is maintained to avoid being misread as is not (generated and maintained).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

+1, I totally misread this 😁

richardlau left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I'm not sure about putting CI job names in the Makefile. It would put the onus on @nodejs/build to keep them in sync.

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

macOS

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This comment doesn't really convey any additional information than can be inferred from the target name.

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think clean is a standard enough makefile target name that this comment shouldn't be necessary. If it is kept, suggest Remove the artifacts... reword.

joyeecheung Nov 16, 2017
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is mostly to distinguish clean from docclean, distclean, test-addons-clean and friends.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Maybe a note that this target will fail unless NODE_VERSION_IS_RELEASE is #define'd as 1 in src/node_version.h.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@richardlau good catch, though I think this should be documented in the makefile instead.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Would a maintainer need to know that? Seems more relevant for users.

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

available

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

maybe use backticks for `make coverage-test`. I read this wrong on first parse and thought it was just awkward wording

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think rpi is the only exception, I've just been grepping through jenkins config files and it's the only one I can find.

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

now that I've seen this and noticed CONFIG_FLAGS I've gone and edited node-test-commit-linux-fips to not be an exception here, it now does

PYTHON=python FLAKY_TESTS=$FLAKY_TESTS_MODE CONFIG_FLAGS=--openssl-fips=$WORKSPACE/fipscanisterdir $MAKE run-ci -j $JOBS

so it's using run-ci like the rest. I'll do the same for the new jobs I'm building out in node-test-commit-linux-linked that do a similar ./configure override thing.

So the only thing I'd suggest about this option is that you note that it's called by run-ci, we don't actually call it directly anywhere.

_(FYI I also think that node-test-commit-linux-fips wasn't running tests in parallel due to a mis-application of JOBS and -j and I think that should be fixed now, that would explain why it's been taking more time than it should in some instances).

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

fips isn't an exception here, although it does run test-ci directly a second time with FIPS enabled

coverage is another exception that doesn't run this

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

you could note that these -upload tasks are strictly for release builds on release machines only, unless you think that's obvious

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

space needed before (unix-like)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

this should probably be python configure --help

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

also worth listing:

BUILDTYPE
JOBS
CONFIG_FLAGS
BUILD_DOWNLOAD_FLAGS
BUILD_INTL_FLAGS

rvagg commented Nov 13, 2017

Copy link
Copy Markdown
Member

are you restricting your comment lines shorter than 80 chars?

thanks for doing this btw! I hope we can keep it all in sync as it evolves

vsemozhetbyt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Sorry for so many nits)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

#vcbuild.bat -> #vcbuildbat ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

configures and prepares?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

A nit: most of the explanations start with a lower case word, some with an upper case one: Build, Generate, Generate, Enable etc.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think these come direct from the configure help, so any changes should be made there.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

@richardlau Yes.

@vsemozhetbyt Do you want to open a PR to fix the configure script? Or I can do that as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Though these are rather simple fixes, I would rather abstain as I know neither Python nor the format of this file. Sorry(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

profileJavaScript -> profile JavaScript ?
nodejs -> Node.js ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

nodejs -> Node.js ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

avilable -> available

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

benchamarks -> benchmarks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

OpenSSl -> OpenSSL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

docuement -> document

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

triggered -> be triggered

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Remove the and which

If we're going to say this here, is it worth saying something similar for make all?

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

binary -> binaries

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Can be run manually too.

joyeecheung Nov 18, 2017
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Hmm I think it's obvious without saying that all the targets here can be run manually (some needs certain environment variables and some probably shouldn't be run manually though)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I would agree, but if you say:

# Note that this is for developers, it is not run on the CI. See run-ci.

elsewhere, then this line suggests that it's used exclusively by the CI job.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Agreed with @richardlau , don't think we should duplicate the ./configure --help output.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

+1, I totally misread this 😁

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I was compiling Alacritty the other day, and I noticed that their make actually defaults to showing help (alias for make help) rather than doing anything. It actually looks really nice, and it'd be really useful if we could do something similar.

I think a help option to make would be less likely to bitrot than separate documentation.

https://github.com/jwilm/alacritty/blob/245a80078180acc2a0a1addc569c15b77991c1c3/Makefile#L18-L21

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think you can just say python configure for both Unix and Windows.

gibfahn commented Nov 15, 2017

Copy link
Copy Markdown
Member

Personally I prefer to leave most of the actual documentation in the code, whiling leave some pointers and necessary explanations in the guide so it's easier to get started with maintaining them.

💯 on this, nothing worse than slightly out-of-date documentation that you can never quite trust.

joyeecheung changed the title [WIP] doc: add guide on maintaining the build files doc: add guide on maintaining the build files Nov 18, 2017

Copy link
Copy Markdown
Member Author

I think I've addressed most comments. @rvagg @TimothyGu @richardlau @vsemozhetbyt @gibfahn PTAL

Copy link
Copy Markdown
Member Author

joyeecheung commented Nov 20, 2017
edited
Loading

Copy link
Copy Markdown
Member Author

Ping @nodejs/build @rvagg @gibfahn @richardlau for reviews, thanks

Copy link
Copy Markdown
Member Author

richardlau left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Small nit otherwise LGTM.

Comment thread Makefile Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

typo: on?

Copy link
Copy Markdown
Member Author

Rebased. Planning on landing this on Monday if there are no more outstanding reviews. cc @nodejs/collaborators

CI: https://ci.nodejs.org/job/node-test-pull-request/12435/

joyeecheung added a commit that referenced this pull request Jan 9, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
joyeecheung added a commit that referenced this pull request Jan 9, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

joyeecheung commented Jan 9, 2018
edited by MylesBorins
Loading

Copy link
Copy Markdown
Member Author

Landed in c91bd2f...89d8657, thanks!

joyeecheung closed this Jan 9, 2018
MylesBorins pushed a commit that referenced this pull request Jan 10, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jan 10, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
MylesBorins mentioned this pull request Jan 10, 2018
MylesBorins pushed a commit that referenced this pull request Jan 10, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jan 10, 2018
PR-URL: #16975
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

Copy link
Copy Markdown
Contributor

Should this land? It will need a manual backport to 6.x and 8.x

Copy link
Copy Markdown
Member Author

@MylesBorins it's just some doc changes so I would say nope

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Footer

© 2026 GitHub, Inc.

Back | FazBrowse Home | New Git URL