Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM by GutoVeronezi · Pull Request #8041 · apache/cloudstack · GitHub

Description

This PR address issue #8034.

While taking the volume snapshot on KVM, we call the method copySnapshotToPrimaryStorageDir. This method copies the base snapshot file to another directory, to enable the block commit for the delta of the disk right after taking the snapshot and avoid creating chains in the primary storage. However, copying a file (via the operating system) with a backing store will also copy the reference; therefore, it causes problems when the volume is not consolidated yet (which is always executed in KVM in volume migration process). If we create a template from the snapshot, and the deployment destination does not have access to the backing store file, the template will not be usable, as it would have a backing store pointing to the template that generated the volume.

This PR intends to solve this situation by converting the base volume to the snapshot file instead of just copying it.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Bug Severity

BLOCKER
Critical
Major
Minor
Trivial

Screenshots (if appropriate):

How Has This Been Tested?

I created a VM from a template and checked the volume's chain:

VM's XML:

      <source file='/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b' index='2'/>
      <backingStore type='file' index='3'>
        <format type='qcow2'/>
        <source file='/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/bb4b7b6f-5249-4c18-a5f6-dd886683b702'/>
        <backingStore/>
      </backingStore>

Template file info:

qemu-img info -U /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/bb4b7b6f-5249-4c18-a5f6-dd886683b702
image: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/bb4b7b6f-5249-4c18-a5f6-dd886683b702
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 2.6 GiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    refcount bits: 16

Volume file info:

qemu-img info -U /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b
image: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 1.66 GiB
cluster_size: 65536
backing file: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Then, I triggered a manual volume snapshot with the VM still running:

logs

2023-10-04 12:42:57,212 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (agentRequest-Handler-5:null) (logid:ae8045e2) Executing command CreateObjectCommand: [{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-190","size":8589934592,"path":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeId":187,"vmName":"i-2-190-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":187,"deviceId":0,"bytesReadRate":104857600,"bytesWriteRate":104857600,"iopsReadRate":1000,"iopsWriteRate":1000,"cacheMode":"NONE","hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"vmName":"i-2-190-VM","name":"test1","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"wait":0,"bypassHostMaintenance":false}].
2023-10-04 12:42:57,212 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:42:57,224 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:ae8045e2) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 12:42:57,224 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:42:57,294 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:ae8045e2) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (46.07 GB) 49466834944 Available: (26.76 GB) 28734914560
2023-10-04 12:42:57,294 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:ae8045e2) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 12:42:57,295 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:42:57,307 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-5:null) (logid:ae8045e2) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (46.07 GB) 49466834944 Available: (26.76 GB) 28734914560
2023-10-04 12:42:57,362 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) Pool [{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","localPath":"\/mnt\/dc916dc7-cc1c-3f3d-905b-b198daf15a79"}] has enough available size [28734914560] to take volume [{"name":"72a45236-0fc8-48ec-9fff-b6894c48273b","path":"\/mnt\/dc916dc7-cc1c-3f3d-905b-b198daf15a79\/72a45236-0fc8-48ec-9fff-b6894c48273b","size":1777606656}] snapshot.
2023-10-04 12:42:57,369 DEBUG [kvm.resource.LibvirtVMDef] (agentRequest-Handler-5:null) (logid:ae8045e2) Using informed label [hdc] for volume [null].
2023-10-04 12:42:57,729 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) Snapshot [41db256b-d8fa-4350-bc9e-0d62e4e9b176] took [0] seconds to finish.
2023-10-04 12:42:57,731 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) Trying to convert volume [volumeTO[uuid=72a45236-0fc8-48ec-9fff-b6894c48273b|path=72a45236-0fc8-48ec-9fff-b6894c48273b|datastore=PrimaryDataStoreTO[uuid=dc916dc7-cc1c-3f3d-905b-b198daf15a79|name=storage-1-nfs|id=1|pooltype=NetworkFilesystem]]] (/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b) to snapshot [/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/41db256b-d8fa-4350-bc9e-0d62e4e9b176].
2023-10-04 12:42:57,741 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-5:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:42:57,750 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Executing: qemu-img convert -O qcow2 -U --image-opts driver=qcow2,file.filename=/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/41db256b-d8fa-4350-bc9e-0d62e4e9b176 
2023-10-04 12:42:57,753 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Executing while with timeout : 3600000
2023-10-04 12:47:55,770 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Execution is successful.
2023-10-04 12:47:55,773 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) Converted volume [volumeTO[uuid=72a45236-0fc8-48ec-9fff-b6894c48273b|path=72a45236-0fc8-48ec-9fff-b6894c48273b|datastore=PrimaryDataStoreTO[uuid=dc916dc7-cc1c-3f3d-905b-b198daf15a79|name=storage-1-nfs|id=1|pooltype=NetworkFilesystem]]] (from path "/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b") to snapshot [/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/41db256b-d8fa-4350-bc9e-0d62e4e9b176].
2023-10-04 12:47:55,774 DEBUG [resource.wrapper.LibvirtUtilitiesHelper] (agentRequest-Handler-5:null) (logid:ae8045e2) The current Libvirt's version [6000000] supports the flag '--delete' on command 'virsh blockcommit'.
2023-10-04 12:47:55,775 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Executing: /bin/bash -c virsh blockcommit i-2-190-VM vda --base /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/72a45236-0fc8-48ec-9fff-b6894c48273b --active --wait --delete --pivot 
2023-10-04 12:47:55,777 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Executing while with timeout : 3600000
2023-10-04 12:47:56,551 DEBUG [utils.script.Script] (agentRequest-Handler-5:null) (logid:ae8045e2) Execution is successful.
2023-10-04 12:47:56,556 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) Successfully merged snapshot [41db256b-d8fa-4350-bc9e-0d62e4e9b176] into VM [i-2-190-VM] volumeTO[uuid=72a45236-0fc8-48ec-9fff-b6894c48273b|path=72a45236-0fc8-48ec-9fff-b6894c48273b|datastore=PrimaryDataStoreTO[uuid=dc916dc7-cc1c-3f3d-905b-b198daf15a79|name=storage-1-nfs|id=1|pooltype=NetworkFilesystem]] base file.
2023-10-04 12:47:56,557 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-5:null) (logid:ae8045e2) The current Libvirt's version supports the flag '--delete' on command 'virsh blockcommit', we will skip the manually deletion of the unused snapshot file [/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/41db256b-d8fa-4350-bc9e-0d62e4e9b176] as it already was automatically deleted.

...


2023-10-04 12:47:57,968 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (agentRequest-Handler-1:null) (logid:ae8045e2) Executing command CopyCommand: [{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/41db256b-d8fa-4350-bc9e-0d62e4e9b176","volume":{"uuid":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-190","size":8589934592,"path":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeId":187,"vmName":"i-2-190-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":187,"deviceId":0,"bytesReadRate":104857600,"bytesWriteRate":104857600,"iopsReadRate":1000,"iopsWriteRate":1000,"cacheMode":"NONE","hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"vmName":"i-2-190-VM","name":"test1","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/187","volume":{"uuid":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-190","size":8589934592,"path":"72a45236-0fc8-48ec-9fff-b6894c48273b","volumeId":187,"vmName":"i-2-190-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":187,"deviceId":0,"bytesReadRate":104857600,"bytesWriteRate":104857600,"iopsReadRate":1000,"iopsWriteRate":1000,"cacheMode":"NONE","hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.30.140/mnt/secondary-storage","_role":"Image"}},"vmName":"i-2-190-VM","name":"test1","hypervisorType":"KVM","id":13,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"snapshot.backup.to.secondary":"true","fullSnapshot":"true"},"options2":{},"wait":21600,"bypassHostMaintenance":false}].
2023-10-04 12:47:57,968 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:47:57,970 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Attempting to create storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a (NetworkFilesystem) in libvirt
2023-10-04 12:47:57,970 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:47:57,974 WARN  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a was not found running in libvirt. Need to create it.
2023-10-04 12:47:57,974 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Didn't find an existing storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a by UUID, checking for pools with duplicate paths
2023-10-04 12:47:57,977 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Checking path of existing pool f838fe2f-44a9-455f-8f3e-5fe9589f45b0 against pool we want to create
2023-10-04 12:47:57,981 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Checking path of existing pool df4a775d-eb19-495b-a73b-002077b254c1 against pool we want to create
2023-10-04 12:47:57,986 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Checking path of existing pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 against pool we want to create
2023-10-04 12:47:57,990 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Attempting to create storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a
2023-10-04 12:47:57,991 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) <pool type='netfs'>
2023-10-04 12:47:57,991 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) (logid:ae8045e2) Executing: /bin/bash -c mountpoint -q /mnt/ad0eb7cd-40b5-3d48-b83c-604e20b7744a 
2023-10-04 12:47:57,993 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) (logid:ae8045e2) Executing while with timeout : 3600000
2023-10-04 12:47:57,997 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) (logid:ae8045e2) Exit value is 1
2023-10-04 12:47:57,997 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) (logid:ae8045e2) 1
2023-10-04 12:47:58,118 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Trying to fetch storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a from libvirt
2023-10-04 12:47:58,119 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:47:58,128 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Successfully refreshed pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a Capacity: (72.83 GB) 78201749504 Used: (50.15 GB) 53843329024 Available: (22.69 GB) 24358420480
2023-10-04 12:47:58,129 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 12:47:58,129 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:47:58,137 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (48.08 GB) 51625066496 Available: (24.75 GB) 26576683008
2023-10-04 12:47:58,338 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-1:null) (logid:ae8045e2) Executing: /usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/41db256b-d8fa-4350-bc9e-0d62e4e9b176 -n 41db256b-d8fa-4350-bc9e-0d62e4e9b176 -p /mnt/ad0eb7cd-40b5-3d48-b83c-604e20b7744a/snapshots/2/187 -t 41db256b-d8fa-4350-bc9e-0d62e4e9b176 
2023-10-04 12:47:58,342 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-1:null) (logid:ae8045e2) Executing while with timeout : 21600000
2023-10-04 12:50:53,604 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-1:null) (logid:ae8045e2) Execution is successful.
2023-10-04 12:50:54,000 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Attempting to remove storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a from libvirt
2023-10-04 12:50:54,001 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-1:null) (logid:ae8045e2) Looking for libvirtd connection at: qemu:///system
2023-10-04 12:50:54,008 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a has no corresponding secret. Not removing any secret.
2023-10-04 12:50:54,329 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-1:null) (logid:ae8045e2) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a was successfully removed from libvirt.
2023-10-04 12:50:54,332 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:ae8045e2) Seq 30-2203667592667725915:  { Ans: , MgmtId: 90520745551922, via: 30, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/187/41db256b-d8fa-4350-bc9e-0d62e4e9b176","id":"0","quiescevm":"false","physicalSize":"3857645568"}},"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }

After that, I checked the snapshot's file on the secondary storage and identified that it did not have a chain (i.e.: it was consolidated):

qemu-img info -U snapshots/2/187/41db256b-d8fa-4350-bc9e-0d62e4e9b176 
image: snapshots/2/187/41db256b-d8fa-4350-bc9e-0d62e4e9b176
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 3.59 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Then, I created a template from the snapshot and a VM with the template:

VM's root volume info:

qemu-img info -U /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/26bc588d-f1e7-42db-95cd-82bf32597fe6
image: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/26bc588d-f1e7-42db-95cd-82bf32597fe6
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 324 KiB
cluster_size: 65536
backing file: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/f234d832-11d3-4e9a-860a-09e1d6ebeae1
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Template info:

qemu-img info -U /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/f234d832-11d3-4e9a-860a-09e1d6ebeae1
image: /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/f234d832-11d3-4e9a-860a-09e1d6ebeae1
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 3.59 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

With that, it was concluded that the template was a single file, without references to the template of the volume that generated the snapshot.

…g volume snapshots on KVM

I am still working on the unit tests and smoke tests.

Manual tests with a stopped VM will be done soon.

Codecov Report

Merging #8041 (315568a) into 4.18 (29c7b31) will increase coverage by 0.05%.
Report is 64 commits behind head on 4.18.
The diff coverage is 19.30%.

@@             Coverage Diff              @@
##               4.18    #8041      +/-   ##
============================================
+ Coverage     13.02%   13.07%   +0.05%     
- Complexity     9032     9109      +77     
============================================
  Files          2720     2720              
  Lines        257080   257519     +439     
  Branches      40088    40151      +63     
============================================
+ Hits          33476    33667     +191     
- Misses       219400   219621     +221     
- Partials       4204     4231      +27

Files	Coverage Δ
...hestration/service/VolumeOrchestrationService.java	100.00% <ø> (ø)
.../main/java/com/cloud/network/IpAddressManager.java	100.00% <100.00%> (ø)
...ava/com/cloud/network/as/AutoScaleVmProfileVO.java	80.20% <100.00%> (+11.66%)	⬆️
...java/com/cloud/upgrade/DatabaseUpgradeChecker.java	40.89% <100.00%> (+0.64%)	⬆️
...va/com/cloud/upgrade/DatabaseVersionHierarchy.java	85.10% <100.00%> (+1.01%)	⬆️
.../api/command/admin/ratelimit/ResetApiLimitCmd.java	0.00% <ø> (ø)
...oud/hypervisor/kvm/resource/LibvirtConnection.java	0.00% <ø> (ø)
.../hypervisor/kvm/storage/ScaleIOStorageAdaptor.java	10.44% <100.00%> (ø)
...ava/com/cloud/api/commands/StopNetScalerVMCmd.java	0.00% <ø> (ø)
...tungsten/api/command/ListTungstenFabricTagCmd.java	0.00% <ø> (ø)
... and 55 more

... and 9 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

wouldn't it be convenient to throw the CloudRuntimeException(convertResult) instead of in the validate method?

@DaanHoogland I will check if we can refactor this part.

@DaanHoogland
Here we return a String instead of throwing an exception so we can handle the merge of the delta before throwing the exception. We could refactor KVMStorageProcessor#createSnapshot a little bit to work with a try-catch-finally. Though, I think we could do it afterward, so we focus on the issue fixing and reduce the scope of changes in this PR. What do you think?

Co-authored-by: dahn <daan.hoogland@gmail.com>

Thanks for quickly raising a PR @GutoVeronezi left some questions, pl advise when this is ready for review/testing

@blueorangutan package

I did some more testing with stopped VMs and the process is occurring as expected too:

logs

2023-10-04 15:10:15,145 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:84c42edf) Request:Seq 29-7266276523785584974:  { Cmd , MgmtId: 90520745551922, via: 29, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":"1","poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":"2049","url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":"false"}},"name":"ROOT-191","size":"(8.00 GB) 8589934592","path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":"188","vmName":"i-2-191-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"1","id":"188","deviceId":"0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":"1","poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":"2049","url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":"false"}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":"14","quiescevm":"false","physicalSize":"0"}},"wait":"0","bypassHostMaintenance":"false"}}] }
2023-10-04 15:10:15,145 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:84c42edf) Processing command: org.apache.cloudstack.storage.command.CreateObjectCommand
2023-10-04 15:10:15,146 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (agentRequest-Handler-2:null) (logid:84c42edf) Executing command CreateObjectCommand: [{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-191","size":8589934592,"path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":188,"vmName":"i-2-191-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":188,"deviceId":0,"hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":14,"quiescevm":false,"physicalSize":0}},"wait":0,"bypassHostMaintenance":false}].
2023-10-04 15:10:15,148 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:10:15,153 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Can not find KVM connection for Instance: i-2-191-VM, continuing.
2023-10-04 15:10:15,153 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: lxc:///
2023-10-04 15:10:15,153 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) No existing libvirtd connection found. Opening a new one
2023-10-04 15:10:15,158 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Can not find LXC connection for Instance: i-2-191-VM, continuing.
2023-10-04 15:10:15,158 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Can not find a connection for Instance i-2-191-VM. Assuming the default connection.
2023-10-04 15:10:15,159 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:10:15,160 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:84c42edf) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 15:10:15,160 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:10:15,168 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:84c42edf) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (57.49 GB) 61730717696 Available: (15.34 GB) 16471031808
2023-10-04 15:10:15,168 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:84c42edf) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 15:10:15,168 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:10:15,179 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:84c42edf) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (57.49 GB) 61730717696 Available: (15.34 GB) 16471031808
2023-10-04 15:10:15,199 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:84c42edf) Trying to convert volume [volumeTO[uuid=b4ad58de-3f95-43f3-a218-d8693082cc99|path=b4ad58de-3f95-43f3-a218-d8693082cc99|datastore=PrimaryDataStoreTO[uuid=dc916dc7-cc1c-3f3d-905b-b198daf15a79|name=storage-1-nfs|id=1|pooltype=NetworkFilesystem]]] (/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/b4ad58de-3f95-43f3-a218-d8693082cc99) to snapshot [/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026].
2023-10-04 15:10:15,203 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:10:15,212 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) (logid:84c42edf) Executing: qemu-img convert -O qcow2 -U --image-opts driver=qcow2,file.filename=/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/b4ad58de-3f95-43f3-a218-d8693082cc99 /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026 
2023-10-04 15:10:15,215 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) (logid:84c42edf) Executing while with timeout : 3600000
2023-10-04 15:12:28,724 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) (logid:84c42edf) Execution is successful.
2023-10-04 15:12:28,725 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:84c42edf) Converted volume [volumeTO[uuid=b4ad58de-3f95-43f3-a218-d8693082cc99|path=b4ad58de-3f95-43f3-a218-d8693082cc99|datastore=PrimaryDataStoreTO[uuid=dc916dc7-cc1c-3f3d-905b-b198daf15a79|name=storage-1-nfs|id=1|pooltype=NetworkFilesystem]]] (from path "/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/b4ad58de-3f95-43f3-a218-d8693082cc99") to snapshot [/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026].
2023-10-04 15:12:28,728 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:84c42edf) Seq 29-7266276523785584974:  { Ans: , MgmtId: 90520745551922, via: 29, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026","id":"0","quiescevm":"false","physicalSize":"0"}},"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }

...

2023-10-04 15:12:30,090 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:84c42edf) Request:Seq 30-2203667592667727137:  { Cmd , MgmtId: 90520745551922, via: 30, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026","volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":"1","poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":"2049","url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":"false"}},"name":"ROOT-191","size":"(8.00 GB) 8589934592","path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":"188","vmName":"i-2-191-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"1","id":"188","deviceId":"0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":"1","poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":"2049","url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":"false"}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":"14","quiescevm":"false","physicalSize":"0"}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/188","volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":"1","poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":"2049","url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":"false"}},"name":"ROOT-191","size":"(8.00 GB) 8589934592","path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":"188","vmName":"i-2-191-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"1","id":"188","deviceId":"0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.30.140/mnt/secondary-storage","_role":"Image"}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":"14","quiescevm":"false","physicalSize":"0"}},"executeInSequence":"false","options":{"snapshot.backup.to.secondary":"true","fullSnapshot":"true"},"options2":{},"wait":"21600","bypassHostMaintenance":"false"}}] }
2023-10-04 15:12:30,091 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:84c42edf) Processing command: org.apache.cloudstack.storage.command.CopyCommand
2023-10-04 15:12:30,091 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (agentRequest-Handler-3:null) (logid:84c42edf) Executing command CopyCommand: [{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026","volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-191","size":8589934592,"path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":188,"vmName":"i-2-191-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":188,"deviceId":0,"hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":14,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/188","volume":{"uuid":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"dc916dc7-cc1c-3f3d-905b-b198daf15a79","name":"storage-1-nfs","id":1,"poolType":"NetworkFilesystem","host":"192.168.31.140","path":"/mnt/primary-storage","port":2049,"url":"NetworkFilesystem://192.168.31.140/mnt/primary-storage/?ROLE=Primary&STOREUUID=dc916dc7-cc1c-3f3d-905b-b198daf15a79","isManaged":false}},"name":"ROOT-191","size":8589934592,"path":"b4ad58de-3f95-43f3-a218-d8693082cc99","volumeId":188,"vmName":"i-2-191-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":188,"deviceId":0,"hypervisorType":"KVM","directDownload":false,"deployAsIs":false},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.30.140/mnt/secondary-storage","_role":"Image"}},"vmName":"i-2-191-VM","name":"test-snap2","hypervisorType":"KVM","id":14,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"snapshot.backup.to.secondary":"true","fullSnapshot":"true"},"options2":{},"wait":21600,"bypassHostMaintenance":false}].
2023-10-04 15:12:30,092 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:12:30,101 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Can not find KVM connection for Instance: i-2-191-VM, continuing.
2023-10-04 15:12:30,101 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: lxc:///
2023-10-04 15:12:30,101 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) No existing libvirtd connection found. Opening a new one
2023-10-04 15:12:30,105 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Can not find LXC connection for Instance: i-2-191-VM, continuing.
2023-10-04 15:12:30,106 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Can not find a connection for Instance i-2-191-VM. Assuming the default connection.
2023-10-04 15:12:30,106 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:12:30,107 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Attempting to create storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a (NetworkFilesystem) in libvirt
2023-10-04 15:12:30,108 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:12:30,109 WARN  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a was not found running in libvirt. Need to create it.
2023-10-04 15:12:30,110 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Didn't find an existing storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a by UUID, checking for pools with duplicate paths
2023-10-04 15:12:30,111 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Checking path of existing pool f838fe2f-44a9-455f-8f3e-5fe9589f45b0 against pool we want to create
2023-10-04 15:12:30,114 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Checking path of existing pool df4a775d-eb19-495b-a73b-002077b254c1 against pool we want to create
2023-10-04 15:12:30,116 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Checking path of existing pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 against pool we want to create
2023-10-04 15:12:30,118 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Attempting to create storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a
2023-10-04 15:12:30,119 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) <pool type='netfs'>
2023-10-04 15:12:30,119 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:84c42edf) Executing: /bin/bash -c mountpoint -q /mnt/ad0eb7cd-40b5-3d48-b83c-604e20b7744a 
2023-10-04 15:12:30,135 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:84c42edf) Executing while with timeout : 3600000
2023-10-04 15:12:30,140 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:84c42edf) Exit value is 1
2023-10-04 15:12:30,140 DEBUG [utils.script.Script] (agentRequest-Handler-3:null) (logid:84c42edf) 1
2023-10-04 15:12:30,263 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Trying to fetch storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a from libvirt
2023-10-04 15:12:30,264 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:12:30,272 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Successfully refreshed pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a Capacity: (72.83 GB) 78201749504 Used: (60.10 GB) 64529104896 Available: (12.73 GB) 13672644608
2023-10-04 15:12:30,272 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Trying to fetch storage pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 from libvirt
2023-10-04 15:12:30,272 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:12:30,284 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Successfully refreshed pool dc916dc7-cc1c-3f3d-905b-b198daf15a79 Capacity: (72.83 GB) 78201749504 Used: (57.78 GB) 62041096192 Available: (15.05 GB) 16160653312
2023-10-04 15:12:30,299 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:84c42edf) Executing: /usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b /mnt/dc916dc7-cc1c-3f3d-905b-b198daf15a79/snapshots/dfba8a17-a688-4bd6-9c05-5b4ce4342026 -n dfba8a17-a688-4bd6-9c05-5b4ce4342026 -p /mnt/ad0eb7cd-40b5-3d48-b83c-604e20b7744a/snapshots/2/188 -t dfba8a17-a688-4bd6-9c05-5b4ce4342026 
2023-10-04 15:12:30,300 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:84c42edf) Executing while with timeout : 21600000
2023-10-04 15:14:14,703 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-3:null) (logid:84c42edf) Execution is successful.
2023-10-04 15:14:15,028 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Attempting to remove storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a from libvirt
2023-10-04 15:14:15,029 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-3:null) (logid:84c42edf) Looking for libvirtd connection at: qemu:///system
2023-10-04 15:14:15,032 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a has no corresponding secret. Not removing any secret.
2023-10-04 15:14:15,290 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:84c42edf) Storage pool ad0eb7cd-40b5-3d48-b83c-604e20b7744a was successfully removed from libvirt.
2023-10-04 15:14:15,292 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:84c42edf) Seq 30-2203667592667727137:  { Ans: , MgmtId: 90520745551922, via: 30, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/188/dfba8a17-a688-4bd6-9c05-5b4ce4342026","id":"0","quiescevm":"false","physicalSize":"2796552192"}},"result":"true","wait":"0","bypassHostMaintenance":"false"}}] }

Snapshot's info on secondary storage:

$ qemu-img info snapshots/2/188/dfba8a17-a688-4bd6-9c05-5b4ce4342026 
image: snapshots/2/188/dfba8a17-a688-4bd6-9c05-5b4ce4342026
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 2.6 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Thanks for quickly raising a PR @GutoVeronezi left some questions, pl advise when this is ready for review/testing

@rohityadavcloud
The PR can already be reviewed/tested; I will now check if the refactoring @DaanHoogland pointed out is viable and work on the smoke tests.

Just for sanity:
@blueorangutan package

@GutoVeronezi a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7230

@blueorangutan test

@shwstppr a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Thanks @GutoVeronezi for raising the PR.

I've a question here. As I understood with this PR we are now backing up the consolidated snapshot file which is good. How is this PR different from reverting the original PR #5297 where I see lot more changes wrt to the behaviour that had introduced. Do we also need to address any code in other places ?

[SF] Trillian test result (tid-7846)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41732 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8041-t7846-kvm-centos7.zip
Smoke tests completed. 106 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_upgrade_kubernetes_cluster	Failure	559.86	test_kubernetes_clusters.py
test_hostha_enable_ha_when_host_in_maintenance	Error	302.73	test_hostha_kvm.py

LGTM

Deployed a test VM. Stopped it. Took ROOT volume snapshot. Verified the snapshot does not have any backing_file
Deleted the VM. This deleted cached template files on the primary store when storage GC ran.
Created a new template from the snapshot <- Successful
Create a new volume from the snapshot <- Successful
Tried deploying a VM with the new template <- Successful

I've not tested revert operations. I hope they still work fine. @GutoVeronezi

@shwstppr @GutoVeronezi have you tested if the qemu-img conversion happens on the primary storage (where the snapshot file is created) or after it's copied to secondary storage ?

@rohityadavcloud happens on primary store before copy/backup

Thanks @GutoVeronezi for raising the PR.

I've a question here. As I understood with this PR we are now backing up the consolidated snapshot file which is good. How is this PR different from reverting the original PR #5297 where I see lot more changes wrt to the behaviour that had introduced. Do we also need to address any code in other places ?

@harikrishna-patnala
Before #5297, we were doing a full VM snapshot and extracting the volume we wanted to back up to the secondary storage. However, the full VM snapshot is a longstanding process that affects the VM's memory and disks; while this process is running, the VMs get frozen due to memory snapshot, and users are unable to manipulate their VM for a long time (if the amount of memory is huge). The proposal #5124 intention was to improve this process by taking an external disk-only snapshot; this process would still produce a consolidated snapshot; therefore, the final result would be the same.

However, as explained in #8034 (comment), due to a misconception on the implementation of the specific creating snapshot workflow, we have reference to the volume template on the snapshot. This reference only exists if we take snapshots of ROOT volumes that were not migrated yet; if we take a snapshot of a ROOT volume that was already migrated at some point in time, the migration process would already have consolidated it (the volume), and we would have a consolidated snapshot. Also, DATADISK volumes do not have a reference to a template; therefore, we would not have a problem in this situation as well. This PR is fixing this specific case by using qemu-img convert instead of copying the base volume to anothery directory in this specific workflow, which is different from reverting the PR (issue #5124 presents in detail the full scope of PR #5297).

@shwstppr @GutoVeronezi have you tested if the qemu-img conversion happens on the primary storage (where the snapshot file is created) or after it's copied to secondary storage ?

@rohityadavcloud, as @shwstppr reported in #8041 (comment), it happens on the primary storage. To better ilustrate the changes, following is an overview of the current workflow and the workflow with the fix, with the actual changes highlighted:

clgtm

extra extra
@blueorangutan test

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Just for the record, following there is a quick use case to reproduce the situation with the current patch and to validate the PR:

create a cluster (c1) with a cluster-wide primary storage (cw-pri-c1); create another cluster (c2) with a cluster-wide primary storage (cw-pri-c2);
create a VM (VM-A) with template on c1 (VM's ROOT volume must be created in storage cw-pri-c1);
create a snapshot (m-snap-1) of the ROOT volume of VM-A;
create a template (tmpl-from-snap) from the snapshot m-snap-1;
create a VM with template tmpl-from-snap on c2.

With the current patch, step 5 is expected to fail, as hosts in c2 does not have access to cw-pri-c1 and template tmpl-from-snap will have a backing store reference to the template file in cw-pri-c1. With the PR, step 5 is expected to finish successfully, as tmpl-from-snap will not have references to any other file.

[SF] Trillian test result (tid-7861)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42465 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8041-t7861-kvm-centos7.zip
Smoke tests completed. 107 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_upgrade_kubernetes_cluster	Failure	566.56	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	Failure	881.73	test_kubernetes_clusters.py

Thanks @GutoVeronezi for raising the PR.
I've a question here. As I understood with this PR we are now backing up the consolidated snapshot file which is good. How is this PR different from reverting the original PR #5297 where I see lot more changes wrt to the behaviour that had introduced. Do we also need to address any code in other places ?

@harikrishna-patnala Before #5297, we were doing a full VM snapshot and extracting the volume we wanted to back up to the secondary storage. However, the full VM snapshot is a longstanding process that affects the VM's memory and disks; while this process is running, the VMs get frozen due to memory snapshot, and users are unable to manipulate their VM for a long time (if the amount of memory is huge). The proposal #5124 intention was to improve this process by taking an external disk-only snapshot; this process would still produce a consolidated snapshot; therefore, the final result would be the same.

However, as explained in #8034 (comment), due to a misconception on the implementation of the specific creating snapshot workflow, we have reference to the volume template on the snapshot. This reference only exists if we take snapshots of ROOT volumes that were not migrated yet; if we take a snapshot of a ROOT volume that was already migrated at some point in time, the migration process would already have consolidated it (the volume), and we would have a consolidated snapshot. Also, DATADISK volumes do not have a reference to a template; therefore, we would not have a problem in this situation as well. This PR is fixing this specific case by using qemu-img convert instead of copying the base volume to anothery directory in this specific workflow, which is different from reverting the PR (issue #5124 presents in detail the full scope of PR #5297).

Thanks for explaining @GutoVeronezi

…taking volume snapshots on KVM (apache#8041) (cherry picked from commit 9b8eaee) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Hello guys,

Just for the record, I reproduced the issue without the patch and validated the changes in this PR. Following there is the tests that I have made and its details:

First, I created two clusters as described here: c1 with a cluster-wide primary storage (cw-pri-c1) and c2 with a cluster-wide primary-storage (cw-pri-c2).

Before the patch

I created the VM VM-A in c1 from a template;
I verified that the ROOT volume of VM-A referenced a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/7c354408-3fed-4113-92b1-4f59f6f97ce0
image: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/7c354408-3fed-4113-92b1-4f59f6f97ce0
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 324 KiB
cluster_size: 65536
backing file: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I created the snapshot vm-a-snap from the ROOT volume of VM-A;
I verified that the snapshot vm-a-snap referenced a backing file:

qemu-img info /mnt/secondary-storage/snapshots/2/275/165748b5-6ac3-4f95-aac5-3df84945074c 
image: /mnt/secondary-storage/snapshots/2/275/165748b5-6ac3-4f95-aac5-3df84945074c
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 22.9 MiB
cluster_size: 65536
backing file: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I created the template tmpl-vm-a from the snapshot vm-a-snap;
I tried to deploy a VM with template tmpl-vm-a on c2;
I verified that the deployment failed.

After applying the patch

I created the VM VM-B in c1 from a template;
I verified that the ROOT volume of VM-B referenced a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/cee7704d-07cc-452f-b6eb-7342ccd87f2c
image: cee7704d-07cc-452f-b6eb-7342ccd87f2c
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 25.9 MiB
cluster_size: 65536
backing file: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I created the snapshot vm-b-snap from the ROOT volume of VM-B;
I verified that the snapshot vm-b-snap was consolidated:

qemu-img info -U /mnt/secondary-storage/snapshots/2/268/60e80be4-ec59-41dc-aaeb-915abe08ec38
image: snapshots/2/268/60e80be4-ec59-41dc-aaeb-915abe08ec38
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 2.6 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I created the template tmpl-vm-b from the snapshot vm-b-snap;
I tried to deploy a VM with template tmpl-vm-b on c2;
I verified that the deployment finished successfully.

I also tested the consolidation of volumes after migrating them, described here. The results were the same before and after applying the patch.

Before the patch

I verified that the ROOT volume of VM-A referenced a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/7c354408-3fed-4113-92b1-4f59f6f97ce0
image: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/7c354408-3fed-4113-92b1-4f59f6f97ce0
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 324 KiB
cluster_size: 65536
backing file: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I added a DATADISK to VM-A;
I verified that the DATADISK did not reference a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/a61c6c88-73ae-4597-be3d-e489b77db09d
image: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/a61c6c88-73ae-4597-be3d-e489b77db09d
file format: qcow2
virtual size: 5 GiB (5368709120 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I migrated VM-A to c2;
I verified that the ROOT volume of VM-A had been consolidated:

qemu-img info -U /mnt/e0560866-421e-313d-8602-320c58842a7e/458e15ce-61cd-476a-88da-660c28c0e940
image: /mnt/e0560866-421e-313d-8602-320c58842a7e/458e15ce-61cd-476a-88da-660c28c0e940
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 2.6 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

And that the DATADISK did not change:

qemu-img info -U /mnt/e0560866-421e-313d-8602-320c58842a7e/bfdc7820-d090-4622-b9aa-f7c780a96ddc
image: /mnt/e0560866-421e-313d-8602-320c58842a7e/bfdc7820-d090-4622-b9aa-f7c780a96ddc
file format: qcow2
virtual size: 5 GiB (5368709120 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

After applying the patch

I verified that the ROOT volume of VM-B referenced a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/cee7704d-07cc-452f-b6eb-7342ccd87f2c
image: cee7704d-07cc-452f-b6eb-7342ccd87f2c
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 25.9 MiB
cluster_size: 65536
backing file: /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/bb4b7b6f-5249-4c18-a5f6-dd886683b702
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I added a DATADISK to VM-B;
I verified that the DATADISK did not reference a backing file:

qemu-img info -U /mnt/c3f4c767-72fe-39e0-a4d1-176c7c19dc74/4bc334f5-95e7-43c6-bede-245f22ef7aa5
image: 4bc334f5-95e7-43c6-bede-245f22ef7aa5
file format: qcow2
virtual size: 5 GiB (5368709120 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

I migrated VM-B to c2;
I verified that the ROOT volume of VM-B had been consolidated:

qemu-img info -U /mnt/e0560866-421e-313d-8602-320c58842a7e/7491cf25-7364-4f6a-bf62-b0f1dca16e61
image: 7491cf25-7364-4f6a-bf62-b0f1dca16e61
file format: qcow2
virtual size: 8 GiB (8589934592 bytes)
disk size: 2.6 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

And that the DATADISK did not change:

qemu-img info -U /mnt/e0560866-421e-313d-8602-320c58842a7e/5776344c-1e74-4653-9fb1-ca10cdca3ab6
image: /mnt/e0560866-421e-313d-8602-320c58842a7e/5776344c-1e74-4653-9fb1-ca10cdca3ab6
file format: qcow2
virtual size: 5 GiB (5368709120 bytes)
disk size: 196 KiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

Finally, I tested all the workflows described in the original PR #5297 twice, once with the global setting snapshot.backup.to.secondary as true and once as false, and verified they are working as expected.

Hello everyone! Sorry to open a discussion on already merged PR, but did somebody test it with encrypted volumes?

…taking volume snapshots on KVM (apache#8041) (cherry picked from commit 9b8eaee) Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Convert volume to another directory instead of copying it while takin…

4dedf49

…g volume snapshots on KVM

boring-cyborg Bot added the component:kvm label Oct 4, 2023

GutoVeronezi mentioned this pull request Oct 4, 2023

KVM volume snapshots are not full snapshots anymore #8034

Closed

GutoVeronezi self-assigned this Oct 4, 2023

GutoVeronezi changed the base branch from main to 4.18 October 4, 2023 13:25

GutoVeronezi added this to the 4.18.2.0 milestone Oct 4, 2023

GutoVeronezi requested review from DaanHoogland, shwstppr, weizhouapache and yadvr October 4, 2023 13:27

DaanHoogland reviewed Oct 4, 2023

View reviewed changes

GutoVeronezi and others added 2 commits October 4, 2023 10:41

Fix unit tests

af83477

Rename variable

315568a

Co-authored-by: dahn <daan.hoogland@gmail.com>

yadvr reviewed Oct 4, 2023

View reviewed changes

Comment thread plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/KVMStorageProcessor.java Resolved

harikrishna-patnala reviewed Oct 5, 2023

View reviewed changes

shwstppr approved these changes Oct 5, 2023

View reviewed changes

DaanHoogland added the Severity:BLOCKER label Oct 5, 2023

DaanHoogland approved these changes Oct 5, 2023

View reviewed changes

DaanHoogland merged commit 9b8eaee into apache:4.18 Oct 6, 2023

GutoVeronezi mentioned this pull request Oct 16, 2023

KVM disk-only based snapshot of volumes instead of taking VM's full snapshot and extracting disks #5124

Closed

-                      } catch (IOException ex) {
-                          return String.format("Unable to copy %s snapshot [%s] to [%s] due to [%s].", volume, baseFile, snapshotPath, ex.getMessage());
+                      } catch (QemuImgException | LibvirtException ex) {
+                          return String.format("Failed to convert %s snapshot of volume [%s] to [%s] due to [%s].", volume, baseFile, snapshotPath, ex.getMessage());

Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM by GutoVeronezi · Pull Request #8041 · apache/cloudstack · GitHub

Repository navigation

Fix: Convert volume to another directory instead of copying it while taking volume snapshots on KVM - #8041

Conversation

GutoVeronezi commented Oct 4, 2023

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

GutoVeronezi commented Oct 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

DaanHoogland Oct 4, 2023

Choose a reason for hiding this comment

Uh oh!

GutoVeronezi Oct 4, 2023

Choose a reason for hiding this comment

Uh oh!

GutoVeronezi Oct 5, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yadvr commented Oct 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yadvr commented Oct 4, 2023

Uh oh!

GutoVeronezi commented Oct 4, 2023

Uh oh!

blueorangutan commented Oct 4, 2023

Uh oh!

blueorangutan commented Oct 4, 2023

Uh oh!

shwstppr commented Oct 4, 2023

Uh oh!

blueorangutan commented Oct 4, 2023

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 5, 2023

Uh oh!

shwstppr left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr commented Oct 5, 2023

Uh oh!

shwstppr commented Oct 5, 2023

Uh oh!

GutoVeronezi commented Oct 5, 2023

Uh oh!

GutoVeronezi commented Oct 5, 2023

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Oct 5, 2023

Uh oh!

blueorangutan commented Oct 5, 2023

Uh oh!

GutoVeronezi commented Oct 5, 2023

Uh oh!

blueorangutan commented Oct 6, 2023

Uh oh!

harikrishna-patnala commented Oct 6, 2023

Uh oh!

winterhazel commented Nov 1, 2023

Uh oh!

slavkap commented Mar 27, 2024

Uh oh!

Reviewers

GutoVeronezi commented Oct 4, 2023 •
edited
Loading

codecov Bot commented Oct 4, 2023 •
edited
Loading

yadvr commented Oct 4, 2023 •
edited
Loading

shwstppr left a comment •
edited
Loading