FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Create new Instance from VM backup by abh1sar · Pull Request #10140 · apache/cloudstack · GitHub

  1. apache
  2. cloudstack

Repository navigation

Create new Instance from VM backup - #10140

Merged
sureshanaparti merged 188 commits into
apache:mainfrom
shapeblue:instance-from-backup
Jul 31, 2025
ConversationCommits (188)ChecksFiles changed
Merged

Create new Instance from VM backup#10140
sureshanaparti merged 188 commits into
apache:mainfrom
shapeblue:instance-from-backup

Conversation

abh1sar commented Dec 25, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Doc PR: apache/cloudstack-documentation#474

Description

This PR adds the ability to create a new Instance from a VM backup for Dummy, NAS and Veeam backup Providers.
This will still work if the original Instance used to create the backup was expunged.
Design doc : https://cwiki.apache.org/confluence/display/CLOUDSTACK/Create+Instance+from+Backup

Other Miscellaneous changes

  1. Framework level enhancements
    a. Set name and description for backups
    b. Show interval type for backup schedule in UI
    c. Improve backup search in ui by keyword
  2. NAS backup provider enhancements
    a. Improved error logging and handling
    b. Quiesce instance option during backup create
  3. Backup And Object storage capacity tracking and alerts.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

1. Create Instance from Backup

Test Case Notes
Create VM from backup using default metadata (no overrides) Validated for all providers
Create VM from backup with custom service offering Service Offering used and Disk size validation enforced
Create VM from backup with custom data volume sizes Fails if smaller than original
Create VM with new IP/MAC (original instance not expunged) New IP/MAC allocated
Create VM with original IP/MAC (original instance expunged) Only works if IP is unassigned
Create VM from backup having ISO Verified that ISO is detached before boot
Create VM from backup having ISO with multiple data volumes Verified volume configurations
Create VM with SSH Key SSH-key access is there
Create VM with multiple data volumes All volumes created and attached
Create VM from backup using default metadata after deleting template Proper error is thrown
Create VM from backup using default metadata after disabling service offering Proper error is thrown
Create VM from backup using default metadata after deleting network Proper error is thrown
Create VM from backup using default metadata after disabling disk offering Proper error is thrown

2. Expunge/Unmanage VM with Backups

Test Case Notes
Remove backup offering from VM with backups Offering removed, backups retained
Expunge VM with retained backups VM deleted, backups remain usable
Attempt to delete backup after expunging source VM No dependency on VM existence
Create VM from backup of an expunged instance Created as expected
Create VM from backup of an unmanaged instance Created as expected
Reassign new backup offering to the VM and trigger new backup New backups tracked under new offering
Verify database entry for expunged VM is retained Entry present as expected

3. Usage Tracking

Test Case Notes
Usage tracked per (VM ID, Backup Offering ID) Validated in usage records
Metrics returned per tuple for all providers (NAS, Veeam, Dummy) As expected
Backup offering removal does not delete usage record if backups exist Usage still visible in usage_vm_backup table
Usage record deleted only when last backup for offering is deleted Verified record purged post-delete
Usage metrics for expunged VMs retained and reported As expected

4. UI Validation

Test Case Notes
“Create Instance from Backup” button appears in backup list UI shows proper dialog
Pre-filled config from backup metadata when clicked on Configure All fields populated correctly
Volume details and limits correctly shown and enforced Volume size limits shown
Use IP Addresses from Backup switch appears only when original VM is expunged As expected
Multiple data volumes rendered correctly in UI form As expected

5. Regression Tests

Test Case Notes
Standard DeployVM API with different combinations of disk offerings, service offerings and ovverride root disk size Pass
Backup Schedule and Delete flows Pass
Restore Backup Pass
Restore Volume and Attach to Instance Pass
VM lifecycle commands Pass

How did you try to break this feature and the system with this change?

codecov Bot commented Dec 25, 2024 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 40.97744% with 1256 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.35%. Comparing base (70468a6) to head (226e82a).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...loudstack/api/command/user/vm/BaseDeployVMCmd.java 47.91% 173 Missing and 39 partials ⚠️
...rg/apache/cloudstack/backup/BackupManagerImpl.java 64.09% 111 Missing and 61 partials ⚠️
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java 44.25% 125 Missing and 40 partials ⚠️
...ack/api/command/user/vm/CreateVMFromBackupCmd.java 6.15% 60 Missing and 1 partial ⚠️
...rg/apache/cloudstack/backup/NASBackupProvider.java 34.40% 58 Missing and 3 partials ⚠️
...rg/apache/cloudstack/backup/dao/BackupDaoImpl.java 21.81% 42 Missing and 1 partial ⚠️
.../apache/cloudstack/backup/DummyBackupProvider.java 0.00% 42 Missing ⚠️
...che/cloudstack/backup/NetworkerBackupProvider.java 0.00% 35 Missing ⚠️
.../apache/cloudstack/backup/VeeamBackupProvider.java 5.88% 32 Missing ⚠️
...a/org/apache/cloudstack/backup/BackupDetailVO.java 18.91% 29 Missing and 1 partial ⚠️
... and 47 more
Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #10140      +/-   ##
============================================
+ Coverage     17.17%   17.35%   +0.17%     
- Complexity    15010    15186     +176     
============================================
  Files          5869     5883      +14     
  Lines        521710   524302    +2592     
  Branches      63509    63962     +453     
============================================
+ Hits          89614    90974    +1360     
- Misses       422039   423051    +1012     
- Partials      10057    10277     +220
Flag Coverage Δ
uitests 3.63% <ø> (-0.13%) ⬇️
unittests 18.38% <40.97%> (+0.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

leo79901 commented Dec 31, 2024

Copy link
Copy Markdown

Yes, we need this !

abh1sar commented Jan 6, 2025

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

blueorangutan commented Jan 6, 2025

Copy link
Copy Markdown

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan commented Jan 6, 2025

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11996

github-actions Bot commented Jan 8, 2025

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

sureshanaparti commented Jul 30, 2025

Copy link
Copy Markdown
Contributor

@blueorangutan package

blueorangutan commented Jul 30, 2025

Copy link
Copy Markdown

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

sureshanaparti commented Jul 30, 2025

Copy link
Copy Markdown
Contributor

@blueorangutan ui

blueorangutan commented Jul 30, 2025

Copy link
Copy Markdown

@sureshanaparti a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan commented Jul 30, 2025

Copy link
Copy Markdown

UI build: ✔️
Live QA URL: https://qa.cloudstack.cloud/simulator/pr/10140 (QA-JID-703)

blueorangutan commented Jul 30, 2025

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14457

sureshanaparti commented Jul 30, 2025

Copy link
Copy Markdown
Contributor

@blueorangutan test

blueorangutan commented Jul 30, 2025

Copy link
Copy Markdown

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

github-actions Bot commented Jul 30, 2025

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

blueorangutan commented Jul 31, 2025

Copy link
Copy Markdown

[SF] Trillian test result (tid-13974)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 80618 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13974-kvm-ol8.zip
Smoke tests completed. 138 look OK, 8 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestRAMCPUResourceAccounting>:setup Error 0.00 test_resource_accounting.py
test_01_list_sec_storage_vm Failure 0.05 test_ssvm.py
test_03_ssvm_internals Failure 0.05 test_ssvm.py
test_05_stop_ssvm Failure 0.04 test_ssvm.py
test_07_reboot_ssvm Failure 0.05 test_ssvm.py
test_11_destroy_ssvm Failure 917.32 test_ssvm.py
ContextSuite context=TestVMWareStoragePolicies>:setup Error 0.00 test_storage_policy.py
test_01_create_template Error 1.14 test_templates.py
test_CreateTemplateWithDuplicateName Error 1.11 test_templates.py
test_02_create_template_with_checksum_sha1 Error 65.65 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.63 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.64 test_templates.py
test_05_create_template_with_no_checksum Error 65.63 test_templates.py
ContextSuite context=TestTemplates>:setup Error 240.85 test_templates.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
test_01_snapshot_usage Error 1.19 test_usage.py
test_01_template_usage Error 1.20 test_usage.py
test_01_volume_usage Error 130.86 test_usage.py
test_06_download_detached_volume Error 300.10 test_volumes.py
test_13_migrate_volume_and_change_offering Error 126.36 test_volumes.py
ContextSuite context=TestIpv6Vpc>:setup Error 0.00 test_vpc_ipv6.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py

sureshanaparti commented Jul 31, 2025

Copy link
Copy Markdown
Contributor

@abh1sar can you check the conflicts

abh1sar commented Jul 31, 2025

Copy link
Copy Markdown
Contributor Author

@blueorangutan package

blueorangutan commented Jul 31, 2025

Copy link
Copy Markdown

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

abh1sar mentioned this pull request Jul 31, 2025
Closed

abh1sar commented Jul 31, 2025

Copy link
Copy Markdown
Contributor Author
  • Tried to create VM from backup and delete the backup before the process ends, the deletion was allowed but the VM was not restored. I think that these processes should be either blocked from executing at the same time, or be queued.

The processes are queued once they reach the agent. i.e, if the new instance is in Restoring state then backup is deleted, the agent will process the restore first and then backup delete. If the new instance is still being created/starting/stopping before restore, and the backup is deleted, delete is not blocked and restore will throw an error that backup is deleted. IMO this is not a must have, and I'll be happy to fix this as a future improvement.

Could we map it as an issue then?

Created issue #11356

blueorangutan commented Jul 31, 2025

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14488

sureshanaparti commented Jul 31, 2025

Copy link
Copy Markdown
Contributor

@blueorangutan test

blueorangutan commented Jul 31, 2025

Copy link
Copy Markdown

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

abh1sar mentioned this pull request Jul 31, 2025
Draft
1 of 13 tasks

blueorangutan commented Aug 1, 2025

Copy link
Copy Markdown

[SF] Trillian test result (tid-13987)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 52137 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10140-t13987-kvm-ol8.zip
Smoke tests completed. 145 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_list_sec_storage_vm Failure 0.05 test_ssvm.py
test_03_ssvm_internals Failure 0.04 test_ssvm.py
test_05_stop_ssvm Failure 0.04 test_ssvm.py
test_11_destroy_ssvm Error 3.23 test_ssvm.py
test_12_destroy_cpvm Error 3.19 test_ssvm.py

abh1sar mentioned this pull request Sep 2, 2025
Merged
2 of 14 tasks
sureshanaparti mentioned this pull request Sep 24, 2025
Closed
sureshanaparti mentioned this pull request Jan 29, 2026
Open
2 of 14 tasks
winterhazel mentioned this pull request Apr 20, 2026
Merged
2 of 12 tasks
bernardodemarco mentioned this pull request Nov 11, 2026
Open
2 of 12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

DaanHoogland DaanHoogland left review comments
weizhouapache weizhouapache left review comments
shwstppr shwstppr approved these changes
rajujith rajujith approved these changes
RosiKyu RosiKyu approved these changes
Pearl1594 Awaiting requested review from Pearl1594
JoaoJandre Awaiting requested review from JoaoJandre

Assignees

No one assigned

Labels

component:api component:backup component:compute component:kubernetes component:kvm component:orchestration component:storage component:UI status:waiting-for-smoketest-results type:new-feature

Projects

No open projects
Apache CloudStack 4.21.0
Status: Done

Milestone

4.21.0

Development

Successfully merging this pull request may close these issues.

Back | FazBrowse Home | New Git URL