Repository navigation
kvm: fill the template's holes before taking its LUKS clone snapshot - #14656
Open
calvix wants to merge 2 commits into
Open
calvix wants to merge 2 commits into
calvix wants to merge 2 commits into
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
An encrypted root is a librbd CoW clone of a plaintext template carrying a LUKS2 header. librbd serves the ranges the clone's own objects do not hold from that parent, as plaintext, which is what makes the inherited filesystem readable. That stops the moment an object exists in the clone: one 4 KiB guest write copies up the whole 4 MiB object, and the ranges inside it that the parent never materialised are from then on read through the crypto layer - where zeros decrypted with AES-XTS are not zeros. Before taking the snapshot encrypted roots are cloned from, rewrite the template image from its own cloudstack-base-snap with qemu-img convert -S 0, so the snapshot has no holes left. The snapshot gets a new name, -luks2, so templates prepared before get a dense one too; roots already cloned from the old -luks snapshot keep it. If another host prepares the same template at the same time, its protected snapshot is used instead of failing the deploy. QemuImg grows setWriteZeroRanges() for the -S 0 this needs, since qemu-img skips the source's zero ranges by default.
Create the base snapshot for encrypted-first deployments and pass the caller timeout to the dense template rewrite. Cover old snapshot upgrades, reuse, failure and retry, and duplicate-snapshot handling.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Encrypted RBD roots created by #13556 can develop filesystem corruption on their first boot when the parent template contains sparse ranges.
Reproduced with an Ubuntu 26.04 template on KVM + Ceph. The guest reported:
e2fsck -fn confirmed corruption on /boot. Some instances also suffered root filesystem errors and remounted it read-only.
The problem happens when a guest write triggers copy-up of a parent object containing holes. Those ranges can return garbage through the clone’s encryption layer instead of the original zeros. In this case, a hole contained an ext4 block bitmap, and writing the superblock in the same object triggered the corruption.
Fix
Before creating the encrypted-clone snapshot, rewrite the template from cloudstack-base-snap using qemu-img convert -n -S 0 to write out its sparse ranges. Create the base snapshot first if the template has never been used for a plaintext clone. The rewrite receives the caller's timeout.
Use a new snapshot name, cloudstack-base-snap-luks2, so templates prepared by the old code are prepared again. Encrypted roots remain thin clones; plaintext roots continue using cloudstack-base-snap.
This fully allocates the template once per pool. It prevents corruption in new roots but does not repair existing ones.
How Has This Been Tested?
RbdEncryptionTest, LibvirtStorageAdaptorTest and KVMStorageProcessorTest pass: 90 tests, no failures or skips. Checkstyle reports no violations.
The regression tests cover encrypted-first deployment, upgrading the old -luks snapshot, reusing an already-prepared snapshot, failed conversion and retry, duplicate-snapshot handling, snapshot creation failure, dense-copy options and timeout propagation.
Duplicate-snapshot handling is tested with mocks; it does not prove simultaneous preparation works. QemuImgTest includes a real dense-allocation test, which requires qemu-img and libvirt and was not run in this validation environment.
testing used the original dense-copy change backported to 4.23 on a three-node KVM + RBD cluster: