[4.16] systemvm: configurable root disk size based on new systemvm template definition by ustcweizhou · Pull Request #3900 · apache/cloudstack · GitHub

Description

The official systemvm template has very few free spaces, it would be better to make the disk size configuration.

Main changes in systemvm template
(1) remove /var and /tmp partitions
(2) move / partition to the last.
(3) install cloud-guest-utils to support "growpart"
(4) install haproxy 1.8 from stretch-backports instead of haproxy 1.7

other changes
(3) add global setting systemvm.root.disk.size
(4) when systemvm starts, allocate all free spaces on disk to the extend partition, root partition (/dev/vda6) and file system.

This applies on all systemvms (cpvm,ssvm,virtual routers, etc)

This fixes #3455

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

build new systemvm template
register new systemvm template
change type from "USER" to "SYSTEM"
change global setting systemvm.root.disk.size to 3GB
destroy ssvm or cpvm
new ssvm/cpvm will have 3GB disk (default 1.5GB) and 2.5 GB free space (default 1GB) on filesystem.

old VR

root@r-269-VM:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              103748       0    103748   0% /dev
tmpfs              47352   23480     23872  50% /run
/dev/vda5        2086316 1209892    860040  59% /
tmpfs             118372       0    118372   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             118372       0    118372   0% /sys/fs/cgroup
/dev/vda1          93207   46230     46977  50% /boot
tmpfs              23672       0     23672   0% /run/user/0

new VR

root@r-270-VM:~# df
Filesystem     1K-blocks   Used Available Use% Mounted on
udev              103688      0    103688   0% /dev
tmpfs              47352   5556     41796  12% /run
/dev/vda6        2687084 970912   1699788  37% /
tmpfs             118372      0    118372   0% /dev/shm
tmpfs               5120      0      5120   0% /run/lock
tmpfs             118372      0    118372   0% /sys/fs/cgroup
/dev/vda1          93207  46361     46846  50% /boot
tmpfs              23672      0     23672   0% /run/user/0

@rhtyd this has some small conflicts with your PR #3797

@rhtyd
if the specified value is small than template size, then it will not be used. "-1" and "0" makes no difference actually.

        Long size = _tmpltMgr.getTemplateSize(template.getId(), vm.getDataCenterId());
        if (rootDisksize != null) {
            rootDisksize = rootDisksize * 1024 * 1024 * 1024;
            if (rootDisksize > size) {
                s_logger.debug("Using root disk size of " + rootDisksize + " Bytes for volume " + name);
                size = rootDisksize;
            } else {
                s_logger.debug("Using root disk size of " + size + " Bytes for volume " + name + "since specified root disk size of " + rootDisksize + " Bytes is smaller than template");
            }
        }

https://github.com/apache/cloudstack/blob/master/engine/orchestration/src/main/java/org/apache/cloudstack/engine/orchestration/VolumeOrchestrator.java#L713-L722

hm, I think you want milestone 4.14 for this one as well, right @weizhouapache ?

hm, I think you want milestone 4.14 for this one as well, right @weizhouapache ?

@DaanHoogland I expect to add to 4.14. However, it require to build new systemvm template (and test it).
4.15 is also good for me.

Thanks per your comment assigned against 4.15 @weizhouapache

@ustcweizhou do we want this for master/4.15+? Can you fix conflicts?

bump @ustcweizhou - can you fix conflicts so we can progress into testing please.

code looks good, we need this integration tested on the new release (which will happen automatically 🤞 )

@rhtyd @DaanHoogland any chance to get this into 4.15 ?

@ustcweizhou should there be a check for > 0 ?

@rhtyd added check.

@ustcweizhou have you tested this for KVM, and non-KVM hypervisors? It's possible the partitions are not called (hard-coded) vda; instead of this can be find/use partitions by mount point names or other valid/static identifiers?

This is screenshot of cpvm on xenserver for example:

Screenshot of partitions of cpvm on kvm:

Screenshot of partitions of cpvm, on vmware:

@ustcweizhou have you tested this for KVM, and non-KVM hypervisors? It's possible the partitions are not called (hard-coded) vda; instead of this can be find/use partitions by mount point names or other valid/static identifiers?

@rhtyd good point. thanks for review

I checked the 4.15 systemvmtemplate, growpart is not available but resize2fs is available. Can we use fdisk instead of growpart to resize partition instead, or some other alternative? This would save from creating a new systemvmtemplate as we get closer to the 4.15 release?

@rhtyd root partition must be the last partition (it is linux swap for now), so this feature requires new systemvm template. let's move to 4.16.
I will work on other hypervisors except kvm.

@ustcweizhou I've left some review comments, please see. As we're getting closer to 4.15 release I would like to avoid anything that could add any regression, usually a systemvmtemplate change could potentially add regressions.

A new Debian 10.6 was released over the weekend so if we build a systemvmtemplate now there may be several differences due to pkg/kernel changes against the one we've been testing for months - http://download.cloudstack.org/systemvm/4.15/

@ustcweizhou I've left some review comments, please see. As we're getting closer to 4.15 release I would like to avoid anything that could add any regression, usually a systemvmtemplate change could potentially add regressions.

A new Debian 10.6 was released over the weekend so if we build a systemvmtemplate now there may be several differences due to pkg/kernel changes against the one we've been testing for months - http://download.cloudstack.org/systemvm/4.15/

@rhtyd ok, then let's not merge this into 4.15
I will fix the issues, hopefully this can be merged into 4.16

old code, needs reconsidering

@rhtyd added check.

ignore my approval ...

@weizhouapache I think @Pearl1594 has already included these changes in her #4329 PR

@weizhouapache I think @Pearl1594 has already included these changes in her #4329 PR

@rhtyd good. I closed this ticket then.

weizhouapache changed the title 4.14 systemvm root disk size systemvm: configurable root disk size based on new systemvm template definition Feb 20, 2020

svenvogel added type:new-feature component:systemvm-template labels Feb 20, 2020

yadvr added this to the 4.15.0.0 milestone Feb 21, 2020

DaanHoogland previously approved these changes Jul 29, 2020

View reviewed changes

yadvr removed this from the 4.15.0.0 milestone Jul 30, 2020

yadvr reviewed Sep 29, 2020

View reviewed changes

yadvr added this to the 4.16.0.0 milestone Oct 7, 2020

weizhouapache changed the title systemvm: configurable root disk size based on new systemvm template definition [DO NOT MERGE] systemvm: configurable root disk size based on new systemvm template definition Oct 30, 2020

ACSGitBot added the type:new_feature label Nov 3, 2020

PaulAngus removed the type:new_feature label Nov 3, 2020

ACSGitBot added the type:new_feature label Nov 8, 2020

DaanHoogland removed the type:new_feature label Nov 10, 2020

DaanHoogland added status:needs-work status:needs-CI status:needs-testing labels Nov 20, 2020

systemvm: configurable root disk size

45828e9

ustcweizhou force-pushed the 4.14-systemvm-root-disk-size branch from 112c8ab to 45828e9 Compare January 17, 2021 09:23

weizhouapache changed the title [DO NOT MERGE] systemvm: configurable root disk size based on new systemvm template definition [4.16] systemvm: configurable root disk size based on new systemvm template definition Jan 19, 2021

systemvm: check systemvm root disk size

9c31a9d

Pearl1594 mentioned this pull request Jan 28, 2021

Adding AutoScaling for cks + CKS CoreOS EOL update + systemvmtemplate improvements #4329

Merged

1 of 5 tasks

weizhouapache approved these changes Jan 28, 2021

View reviewed changes

weizhouapache closed this Mar 23, 2021

weizhouapache mentioned this pull request Jun 4, 2021

systemvmtemplate: use latest LTS kernel from buster-ports #5073

Merged

Pearl1594 mentioned this pull request Aug 9, 2021

Configurable disk size of systemvm #3455

Closed

DaanHoogland removed the status:needs-testing label Jan 28, 2026

+                if which growpart > /dev/null; then
+                  growpart /dev/vda 2
+                  growpart /dev/vda 6
+                  resize2fs /dev/vda6

+                      Long rootDiskSize = rootDiskOfferingInfo.getSize();
+                      if (vm.getType().isUsedBySystem() && SystemVmRootDiskSize.value() != null) {
+                          rootDiskSize = SystemVmRootDiskSize.value();

[4.16] systemvm: configurable root disk size based on new systemvm template definition by ustcweizhou · Pull Request #3900 · apache/cloudstack · GitHub

Repository navigation

[4.16] systemvm: configurable root disk size based on new systemvm template definition - #3900

[4.16] systemvm: configurable root disk size based on new systemvm template definition#3900ustcweizhou wants to merge 2 commits intoapache:masterapache/cloudstack:masterfrom ustcweizhou:4.14-systemvm-root-disk-sizeustcweizhou/cloudstack:4.14-systemvm-root-disk-sizeCopy head branch name to clipboard

Conversation

ustcweizhou commented Feb 20, 2020 • edited by weizhouapache Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

ustcweizhou commented Feb 20, 2020

Uh oh!

weizhouapache commented Feb 20, 2020

Uh oh!

DaanHoogland commented Feb 20, 2020

Uh oh!

weizhouapache commented Feb 20, 2020

Uh oh!

yadvr commented Feb 21, 2020

Uh oh!

yadvr commented Jun 12, 2020

Uh oh!

PaulAngus commented Jul 10, 2020

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Sep 26, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr commented Sep 29, 2020

Uh oh!

weizhouapache commented Sep 29, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Jan 28, 2021

Uh oh!

yadvr commented Mar 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weizhouapache commented Mar 23, 2021

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[4.16] systemvm: configurable root disk size based on new systemvm template definition#3900
ustcweizhou wants to merge 2 commits into
apache:masterfrom
ustcweizhou:4.14-systemvm-root-disk-size

ustcweizhou commented Feb 20, 2020 •
edited by weizhouapache
Loading

yadvr commented Mar 23, 2021 •
edited
Loading