Support VDDK VMware VM migrations to CEPH storage by andrijapanicsb · Pull Request #13270 · apache/cloudstack · GitHub

Companion 4.22 PR: #13294

This PR is intended to keep the main and 4.22 branches aligned for the VDDK-to-RBD import feature.

Description

This PR extends the existing VMware-to-KVM VDDK import flow with Ceph RBD support, while preserving the current VDDK/QCOW2 behaviour.

The existing VDDK/QCOW2 writer remains supported in both existing modes:

With forceconverttopool=true, CloudStack continues to let virt-v2v write QCOW2 output directly into the selected supported primary storage pool using -o local -os <pool-path> -of qcow2.
With forceconverttopool=false, CloudStack continues to use the staged flow: virt-v2v writes finalized QCOW2 disks to temporary conversion storage, and the import step then moves/copies those disks to the final destination pool.

This change adds a destination-aware writer model for VDDK imports. The current file/QCOW2 writer path remains, and a new Ceph RBD/raw writer path is added. This gives the VDDK import flow a cleaner foundation for adding more destination-specific writers later.

For Ceph RBD, two modes are supported:

Direct RBD mode

When usevddk=true, forceconverttopool=true, and convertinstancepoolid points to an RBD primary storage pool, CloudStack can write the VMware disk data directly into raw RBD images.

This path does not use the normal virt-v2v -o local -of qcow2 writer, because RBD is not a local filesystem target. Instead, the conversion host:
- exposes the VMware source disks through VDDK/nbdkit
- writes the data directly into raw RBD images using qemu-img convert
- runs guest finalization in place on the newly-created RBD images using virt-v2v-in-place, or virt-v2v --in-place when supported
The in-place finalization happens only on the destination RBD images. The VMware source disks are not modified.
Staged RBD mode

When usevddk=true and forceconverttopool=false, RBD can still be used as the final destination through the existing staged model.

In this mode:
- regular VDDK/virt-v2v creates finalized QCOW2 disks on temporary conversion storage
- the import step then converts/copies those QCOW2 disks into raw RBD images
This mode does not require in-place virt-v2v, because regular virt-v2v has already finalized the guest while creating the temporary QCOW2 disks.

Direct RBD mode requires newer conversion-host tooling. The practical recommended baseline is EL9-family KVM hosts, such as Oracle Linux 9, Rocky Linux 9, AlmaLinux 9, or RHEL 9, and Ubuntu 24.04-style hosts where virt-v2v-in-place or virt-v2v --in-place is available. EL8 and Ubuntu 22.04-style hosts should use staged RBD import unless compatible in-place tooling has been explicitly installed and detected by CloudStack.

The implementation adds KVM host capability detection for:

VDDK RBD direct import support
in-place virt-v2v support
qemu-img RBD support
qemu-based RBD copy support

Direct RBD mode is rejected early when the selected conversion host does not support the required in-place finalization path. Staged RBD remains available in that case.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

BLOCKER
Critical
Major
Minor
Trivial

Screenshots (if appropriate):

N/A

How Has This Been Tested?

Focused compile and unit test coverage was run for the changed API/core/server/KVM paths, including:

git diff --check
mvn -pl api,core -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl engine/orchestration -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl plugins/hypervisors/kvm -am -DskipTests -Dcheckstyle.skip=true compile
mvn -pl plugins/hypervisors/kvm -am -Dtest=LibvirtCheckConvertInstanceCommandWrapperTest,LibvirtConvertInstanceCommandWrapperTest,LibvirtImportConvertedInstanceCommandWrapperTest -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test
mvn -pl server -am -Dtest=UnmanagedVMsManagerImplTest#testValidateSelectedConversionStoragePoolForVddkSkipsTemporaryPoolForStagedImport+testSelectKVMHostForConversionInClusterDirectRbdAutoSelectsHostWithDirectSupport+testSelectKVMHostForConversionInClusterDirectRbdFailsWithoutDirectSupport+testCheckConversionStoragePoolRbdAllowedForVddkForceConvertToPool -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test

The tests cover direct RBD host capability checks, rejection when direct RBD support is missing, staged RBD destination handling, RBD destination pool type propagation, RBD disk metadata, and server-side validation for RBD storage pools.

How did you try to break this feature and the system with this change?

Negative tests were added for hosts without direct VDDK/RBD support, and staged RBD import is validated separately so that lack of in-place virt-v2v blocks only direct RBD mode, not RBD import as a whole. Existing VDDK/QCOW2 direct and staged flows are kept on the existing writer path unless the selected forced conversion pool is RBD.

Codecov Report

❌ Patch coverage is 26.08696% with 340 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.11%. Comparing base (c0ce5b4) to head (4870053).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../wrapper/LibvirtConvertInstanceCommandWrapper.java	4.02%	187 Missing and 4 partials ⚠️
.../apache/cloudstack/vm/UnmanagedVMsManagerImpl.java	59.04%	37 Missing and 6 partials ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java	0.00%	25 Missing ⚠️
...com/cloud/agent/api/to/VmwareVddkSourceDiskTO.java	0.00%	21 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java	0.00%	18 Missing ⚠️
.../LibvirtImportConvertedInstanceCommandWrapper.java	72.54%	9 Missing and 5 partials ⚠️
.../java/com/cloud/agent/api/to/RemoteInstanceTO.java	0.00%	6 Missing ⚠️
.../org/apache/cloudstack/vm/UnmanagedInstanceTO.java	0.00%	6 Missing ⚠️
...va/com/cloud/agent/api/ConvertInstanceCommand.java	0.00%	6 Missing ⚠️
...loud/agent/api/ImportConvertedInstanceCommand.java	16.66%	5 Missing ⚠️
... and 2 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13270      +/-   ##
============================================
+ Coverage     18.10%   18.11%   +0.01%     
- Complexity    16749    16789      +40     
============================================
  Files          6037     6038       +1     
  Lines        542796   543226     +430     
  Branches      66456    66507      +51     
============================================
+ Hits          98268    98417     +149     
- Misses       433492   433759     +267     
- Partials      11036    11050      +14

Flag	Coverage Δ
uitests	3.51% <ø> (ø)
unittests	19.28% <26.08%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blueorangutan package kvm

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18075

Suggested change

      
                private String vmwareMoref;
          
                private String vmwareMoRef;

?

snapshot internal ref of VMware as reported by vCenter

public String getVmwareMoref() {
return vmwareMoref;
}

public void setVmwareMoref(String vmwareMoref) {
    this.vmwareMoref = vmwareMoref;

"r" is lowewrcase, don't capitalise it - or do it everywher :) ? Or....? You would know better

Suggested change

      
                private String vmwareMoref;
          
                private String vmwareMoRef;

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@blueorangutan test keepEnv

@andrijapanicsb a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[SF] Trillian test result (tid-16214)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 48589 seconds
Marvin logs: [archive removed]
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

@blueorangutan help

@andrijapanicsb [SL] I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv] [qemuEv] [basicZone|securityGroups]
Mgmt OS options: ['suse15', 'alma10', 'ol10', 'rocky10', 'alma9', 'centos7', 'centos6', 'rocky9', 'alma8', 'ubuntu18', 'ol9', 'ol8', 'ubuntu22', 'debian12', 'ubuntu20', 'rocky8', 'ubuntu24']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-rocky9', 'kvm-rocky10', 'kvm-ol8', 'kvm-ol9', 'kvm-ol10', 'kvm-alma8', 'kvm-alma9', 'kvm-alma10', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-ubuntu22', 'kvm-ubuntu24', 'kvm-debian12', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'vmware-80', 'vmware-80u1', 'vmware-80u2', 'vmware-80u3', 'vmware-80u3e', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xenserver-84', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82', 'xcpng83']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
when qemuEv is passed, it will deploy KVM hyperviosr hosts with qemu-kvm-ev, else it will default to stock qemu.
When basicZone and/or securityGroups are passed it will create a zone of the last type specified (default is Advanced)
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'shwstppr', 'Damans227', 'vishesh92', 'Pearl1594', 'harikrishna-patnala', 'nvazquez', 'DaanHoogland', 'weizhouapache', 'borisstoyanov', 'vladimirpetrov', 'kiranchavala', 'andrijapanicsb', 'NuxRo', 'rajujith', 'sureshanaparti', 'abh1sar', 'sudo87', 'RosiKyu']

@blueorangutan package kvm

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 18089

@blueorangutan package kvm

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18094

@blueorangutan package none

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18095

@blueorangutan package kvm

@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress.

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18102

@blueorangutan test ol8 kvm-ol8 keepEnv

@andrijapanicsb a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[SF] Trillian test result (tid-16230)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49566 seconds
Marvin logs: [archive removed]
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

@blueorangutan test ol9 kvm-ol9 keepEnv

@andrijapanicsb a [SL] Trillian-Jenkins test job (ol9 mgmt + kvm-ol9) has been kicked to run smoke tests

[SF] Trillian test result (tid-16234)
Environment: kvm-ol9 (x2), zone: Advanced Networking with Mgmt server ol9
Total time taken: 54398 seconds
Marvin logs: [archive removed]
Smoke tests completed. 151 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

Closing in favour of 4.22 due to forward merge process.

@blueorangutan sweep

Support VDDK imports into RBD storage

bd86d7c

boring-cyborg Bot added component:agent component:api component:kvm component:orchestration component:vmware labels May 28, 2026

andrijapanicsb changed the title Support VDDK imports into RBD storage Support VDDK VMware VM migratiosn to CEPH storage May 28, 2026

andrijapanicsb requested review from harikrishna-patnala, nvazquez and weizhouapache May 28, 2026 07:54

DaanHoogland requested a review from Copilot May 28, 2026 09:59

Copilot started reviewing on behalf of DaanHoogland May 28, 2026 09:59

View session

DaanHoogland reviewed May 28, 2026

View reviewed changes

Copilot AI reviewed May 28, 2026

Avoid null import task update on early import failure

fb6bcea

Clarify direct RBD host capability error

338c28c

andrijapanicsb changed the title Support VDDK VMware VM migratiosn to CEPH storage Support VDDK VMware VM migrations to CEPH storage May 30, 2026

Clarify destination pool tag mismatch errors

d068637

andrijapanicsb mentioned this pull request May 30, 2026

Support VDDK imports into RBD storage #13294

Closed

2 of 14 tasks

Clarify service offering storage tag error

4870053

andrijapanicsb closed this Jun 8, 2026

andrijapanicsb deleted the vddk-rbd-direct-import-main branch July 2, 2026 21:39

weizhouapache added the component:ceph label Aug 2, 2026

                   private String datacenterName;
                   private String clusterName;
                   private String hostName;
+                  private String vmwareMoref;

                   private String bootType;
                   private String bootMode;
+                  private String vmwareMoref;

Support VDDK VMware VM migrations to CEPH storage by andrijapanicsb · Pull Request #13270 · apache/cloudstack · GitHub

Repository navigation

Support VDDK VMware VM migrations to CEPH storage - #13270

Support VDDK VMware VM migrations to CEPH storage#13270andrijapanicsb wants to merge 5 commits intoapache:mainapache/cloudstack:mainfrom andrijapanicsb:vddk-rbd-direct-import-mainandrijapanicsb/cloudstack:vddk-rbd-direct-import-mainCopy head branch name to clipboard

Conversation

andrijapanicsb commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

andrijapanicsb commented May 28, 2026

Uh oh!

blueorangutan commented May 28, 2026

Uh oh!

blueorangutan commented May 28, 2026

Uh oh!

DaanHoogland May 28, 2026

Choose a reason for hiding this comment

Uh oh!

andrijapanicsb May 28, 2026

Choose a reason for hiding this comment

Uh oh!

andrijapanicsb May 28, 2026

Choose a reason for hiding this comment

Uh oh!

DaanHoogland May 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

andrijapanicsb commented May 28, 2026

Uh oh!

blueorangutan commented May 28, 2026

Uh oh!

blueorangutan commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrijapanicsb commented May 29, 2026

Uh oh!

blueorangutan commented May 29, 2026

Uh oh!

andrijapanicsb commented May 29, 2026

Uh oh!

blueorangutan commented May 29, 2026

Uh oh!

blueorangutan commented May 29, 2026

Uh oh!

andrijapanicsb commented May 29, 2026

Uh oh!

blueorangutan commented May 29, 2026

Uh oh!

blueorangutan commented May 29, 2026

Uh oh!

andrijapanicsb commented May 30, 2026

Uh oh!

blueorangutan commented May 30, 2026

Uh oh!

blueorangutan commented May 30, 2026

Uh oh!

andrijapanicsb commented May 30, 2026

Uh oh!

blueorangutan commented May 30, 2026

Uh oh!

blueorangutan commented May 30, 2026

Uh oh!

andrijapanicsb commented May 31, 2026

Uh oh!

blueorangutan commented May 31, 2026

Uh oh!

Support VDDK VMware VM migrations to CEPH storage#13270
andrijapanicsb wants to merge 5 commits into
apache:mainfrom
andrijapanicsb:vddk-rbd-direct-import-main

andrijapanicsb commented May 28, 2026 •
edited
Loading

codecov Bot commented May 28, 2026 •
edited
Loading

blueorangutan commented May 29, 2026 •
edited
Loading

blueorangutan commented May 31, 2026 •
edited
Loading

blueorangutan commented Jun 2, 2026 •
edited
Loading