Repository navigation
kvm: remove the partial RBD image when a template copy fails - #14393
weizhouapache wants to merge 1 commit into
Conversation
Sorry, something went wrong.
When qemu-img convert into an RBD pool failed or timed out, the half-written image was left behind. Every later copy of the same template then failed with "rbd create: File exists", so deployments from that template broke on the pool until the image was removed by hand. - check whether the destination RBD image exists before converting, and refuse to overwrite it, as it may be a complete image used by clones - remove the image again if the copy created it and then failed - do not print the Ceph key in the "Failed to convert" error
|
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Sorry, something went wrong.
Codecov Report❌ Patch coverage is 6.12245% with 46 lines in your changes missing coverage. Please review.
@@ Coverage Diff @@
## 4.22 #14393 +/- ##
=========================================
Coverage 18.01% 18.02%
- Complexity 16265 16272 +7
=========================================
Files 5936 5936
Lines 536395 536442 +47
Branches 65701 65705 +4
=========================================
+ Hits 96658 96695 +37
- Misses 428710 428719 +9
- Partials 11027 11028 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness.
|
Sorry, something went wrong.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19535 |
Sorry, something went wrong.
| logger.error("Failed to convert from {} to RBD image {} the error was: {}", srcFilename, rbdDestPath, maskRbdKey(e.getMessage())); | ||
| newDisk = null; | ||
| if (destCreatedByCopy) { | ||
| removeRbdImageQuietly(destPool, name); |
There was a problem hiding this comment.
what happens on a timeout? qemu-img gets killed, so ceph still sees it on the image for about 30s and the remove here likely fails with "image has watchers"
Sorry, something went wrong.
| String srcFilename = srcFile != null ? srcFile.getFileName() : null; | ||
| String destFilename = destFile != null ? destFile.getFileName() : null; | ||
| logger.error(String.format("Failed to convert from %s to %s the error was: %s", srcFilename, destFilename, e.getMessage())); | ||
| logger.error("Failed to convert from {} to RBD image {} the error was: {}", srcFilename, rbdDestPath, maskRbdKey(e.getMessage())); |
There was a problem hiding this comment.
should the rbd to rbd copy further down get the same fix? it still logs the key and leaves the image behind when it fails
Sorry, something went wrong.
Description
When qemu-img convert into an RBD pool failed or timed out, the half-written image was left behind. Every later copy of the same template then failed with "rbd create: File exists", so deployments from that template broke on the pool until the image was removed by hand.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?