tfchain-validator: unattended installs, safe re-runs, and secrets hygiene - #107
Open
PeterNashaat wants to merge 1 commit into
Open
tfchain-validator: unattended installs, safe re-runs, and secrets hygiene#107PeterNashaat wants to merge 1 commit into
PeterNashaat wants to merge 1 commit into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
…iene - .gitignore for .secrets.env under tfchain-validator/ and tfchain-validator-snapshots/. Every other component already had one; these two are the ones whose .secrets.env holds the node key and the validator wallet mnemonic, and without it a git add -A in a clone stages them. - install-tfchain-validator.sh and validator-init.sh accept -y/--yes (or ASSUME_YES=1) so they can run from CI, Terraform or config management. - the installer no longer restores the snapshot when the node already has a chain database, so re-running it cannot clobber a synced validator. - snapshot download goes to /srv/grid_snapshots_tmp on every network; qanet, testnet and mainnet used the home directory, which can fill the root disk (the mainnet snapshot is ~28G compressed). - validator-init.sh reminds the operator to clear MNEMONIC afterwards. - prep-env-prereq.sh installs node_exporter 1.12.1 instead of 1.6.0. readme: how to move an existing validator to new hardware (keys reused, no council motion, stop the old node first), container log rotation, and the container-storage caveat on VMs whose root filesystem disallows mknod.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While moving a validator to new hardware and provisioning a few more from
Terraform, I ran into a handful of rough edges in tfchain-validator/. All of
these are things that bit me for real:
Secrets hygiene. tfchain-validator/ and tfchain-validator-snapshots/
are the only components without a .gitignore for .secrets.env — and they
are the ones whose .secrets.env holds the node key and the validator wallet
mnemonic. Following the readme leaves that file inside the clone, one
git add -A away from being committed. Added the same .gitignore the other
components already have.
Unattended installs. install-tfchain-validator.sh and
validator-init.sh block on read, so they can't be driven from CI,
Terraform or config management. They now accept -y/--yes (or
ASSUME_YES=1) and are unchanged when run by hand.
Re-running the installer could destroy a synced node. The snapshot restore
ran unconditionally and extracted over the existing database. It now skips the
restore when the chain database is already populated.
Snapshot download location. qanet, testnet and mainnet downloaded the
archive into the home directory, i.e. the root filesystem; devnet used /srv.
The mainnet snapshot is ~28G compressed, which is enough to fill a root disk.
All four now use /srv/grid_snapshots_tmp.
Smaller bits. validator-init.sh reminds the operator to clear MNEMONIC
afterwards, and prep-env-prereq.sh installs node_exporter 1.12.1 instead of
1.6.0.
Readme. Three additions:
node key means no setKeys and no council motion, and the old node must be
stopped first or you are equivocating. That path wasn't documented anywhere.
down for us; log-opts only apply to containers created afterwards.
VM images with a virtiofs root, mknod is denied and image extraction fails
on whiteout conversion. Recent Docker keeps images in the containerd image
store, so moving data-root alone isn't enough; containerd's root has to
move too.
No behaviour changes for anyone running the scripts interactively.