FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Match template tags with host tag for vm deployment by ravening · Pull Request #4362 · apache/cloudstack · GitHub

  1. apache
  2. cloudstack

Repository navigation

Match template tags with host tag for vm deployment - #4362

Closed
ravening wants to merge 3 commits into
apache:mainfrom
ravening:vm-template-tags
ConversationCommits (3)ChecksFiles changed
Closed

Match template tags with host tag for vm deployment#4362
ravening wants to merge 3 commits into
apache:mainfrom
ravening:vm-template-tags

Conversation

ravening commented Oct 1, 2020 •
edited by PaulAngus
Loading

Copy link
Copy Markdown
Member

Description

  1. Match the template tags with host tags for VM deployment/migration
  2. Allow updating templates/isos with template tag by admin
  3. Allow adding template tag while registering/creating template/iso
  4. Allow searching templates/isos using template tags

If a template has a template tag then deploy the VM which uses this
template on hosts that match the template tags. If no suitable
hosts are found then throw an exception

Similarly, match the template tag with host tags while starting or
migrating a VM to a different host

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)

Screenshots (if appropriate):

How Has This Been Tested?

test 1

add different tag to template than hosts
try to deploy. it should fail
try to migrating existing vm. it should fail

test 2

remove all the tags on template
add tags to hosts
try to deploy vm
it can be deployed on any host
try to migrate it.
migration should be possible
stop/start is also possible

test 3

add tag to template
remove tags from host
try to deploy vm
deployment should fail
try to migrate already created vm with template tag
migration should fail

test 4

set same template tag on template and only one host
try to deploy vm
it should deploy vm on host with matching tag

test 5

ow try to migrate above created vm having specific tag
migration should fail as other hosts have either no tags or different tag

test 6

stop the above created vm
try to start of different host
start should fail as the tags dont match

test 7

now remove the tag from the template of the above created vm
try to migrate it
it should work fine

test 8

remove all tags from everyhwere
select any vm and set service offering tag as "ubuntu"
now set host tag as "windows" for all hosts
try to start/deploymigrate vm
it should not start/deploy/migrate as tags dont match

test 9

now set service offering tag as "windows"
try to start/deploy/migrate
it should successfully start/deploy/migrate on/to any host

Same steps for iso as well

ravening mentioned this pull request Oct 1, 2020
Open
PaulAngus removed the type:bug label Nov 7, 2020

PaulAngus commented Nov 7, 2020

Copy link
Copy Markdown
Member

@ravening could you give me some of the use cases for this change please, I'm not quite following.

ravening commented Nov 7, 2020

Copy link
Copy Markdown
Member Author

@ravening could you give me some of the use cases for this change please, I'm not quite following.

@PaulAngus if template has a tag "windows" then the VM's created using this template should be deployed on hosts with tags "windows". Same thing for templates having tags like "linux", "Unix" it any other tag.

So we should not deploy templates having "windows" tag on hosts having "linux" tag

Hope it's clear

shwstppr added this to the 4.16.0.0 milestone Jan 25, 2021

shwstppr commented Jan 25, 2021

Copy link
Copy Markdown
Contributor

@ravening Needs conflicts to be resolved. Moving to 4.16.0.0 milestone based on base branch please update if needed

ravening commented Jan 26, 2021

Copy link
Copy Markdown
Member Author

@ravening Needs conflicts to be resolved. Moving to 4.16.0.0 milestone based on base branch please update if needed

@shwstppr resolved conflicts

weizhouapache commented Mar 16, 2021

Copy link
Copy Markdown
Member

as #4452 has been merged, build will fail.

weizhouapache reopened this Mar 16, 2021

yadvr commented Jun 17, 2021

Copy link
Copy Markdown
Member

ping @ravening conflict fixing please

sureshanaparti commented Jun 25, 2021

Copy link
Copy Markdown
Contributor

@ravening any update on this please

nvazquez commented Aug 12, 2021

Copy link
Copy Markdown
Contributor

Ping @ravening

DaanHoogland commented Aug 20, 2021

Copy link
Copy Markdown
Contributor

good functionality, clear description and sane as far as I can tell.
@blueorangutan package

blueorangutan commented Aug 20, 2021

Copy link
Copy Markdown

@DaanHoogland a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan commented Aug 20, 2021

Copy link
Copy Markdown

Packaging result: ✖️ el7 ✖️ el8 ✖️ debian ✖️ suse15. SL-JID 946

nvazquez commented Sep 3, 2021

Copy link
Copy Markdown
Contributor

Ping @ravening

sureshanaparti commented Sep 15, 2021

Copy link
Copy Markdown
Contributor

ping @ravening, can you fix conflicts

nvazquez commented Sep 16, 2021

Copy link
Copy Markdown
Contributor

@blueorangutan package

blueorangutan commented Sep 16, 2021

Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan commented Sep 16, 2021

Copy link
Copy Markdown

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 1290

DaanHoogland commented Sep 17, 2021

Copy link
Copy Markdown
Contributor

running test_host_tags.py, I get

Test template/host tag ... === TestName: test_01_template_host_tag | Status : EXCEPTION ===
ERROR

======================================================================
ERROR: Test template/host tag
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/marvin/tests/component/test_host_tags.py", line 210, in test_01_template_host_tag
    cls.vm_3.start(cls.apiclient, hostid = host.id)
  File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 722, in start
    apiclient.startVirtualMachine(cmd)
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 4014, in startVirtualMachine
    response = self.connection.marvinRequest(command, response_type=response, method=method)
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest
    raise e
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest
    raise self.__lastError
  File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 105, in __poll
    % async_response)
Exception: Job failed: {accountid : 'e423f3e7-16f6-11ec-a2d3-1e00b900043d', userid : 'e4251d92-16f6-11ec-a2d3-1e00b900043d', cmd : 'org.apache.cloudstack.api.command.admin.vm.StartVMCmdByAdmin', jobstatus : 2, jobprocstatus : 0, jobresultcode : 530, jobresulttype : 'object', jobresult : {errorcode : 431, errortext : 'The virtual machine dd283411-86dd-4b94-866c-3d32dd72a89c (Test VM) is already running'}, jobinstancetype : 'VirtualMachine', jobinstanceid : 'dd283411-86dd-4b94-866c-3d32dd72a89c', created : '2021-09-17T10:44:08+0000', completed : '2021-09-17T10:44:08+0000', jobid : '0e11e39a-3329-4cdd-820f-6f904480b5f1'}

running test_template_tags.py

executing /usr/local/bin/nosetests --with-xunit --xunit-file=/marvin/MarvinLogs/test_template_tags.py.xml --with-marvin --marvin-config=/marvin/pr4362-t2092-kvm-centos7-advanced-cfg --hypervisor=kvm -s -a tags=advanced tests/component/test_template_tags.py

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/Sep_17_2021_11_08_00_IQNIPQ All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_01_update_template_tags | Status : SUCCESS ===

=== TestName: test_02_register_template_with_tags | Status : SUCCESS ===

DaanHoogland commented Sep 17, 2021

Copy link
Copy Markdown
Contributor

I created a template with no tag
I have two hosts
one host as no tags
the other one has a tag "none"
I deploy a VM
result:

2021-09-17 11:26:32,657 DEBUG [c.c.a.ApiServlet] (qtp769798433-15:ctx-9d0ad7da) (logid:30c456f4) ===START===  172.16.250.3 -- GET  jobId=f29d9a58-fd07-4186-bc85-ea2cc60ac1f3&command=queryAsyncJobResult&response=json
2021-09-17 11:26:32,661 DEBUG [c.c.a.ApiServer] (qtp769798433-15:ctx-9d0ad7da ctx-89672168) (logid:30c456f4) CIDRs from which account 'Acct[e423f3e7-16f6-11ec-a2d3-1e00b900043d-admin] -- Account {"id": 2, "name": "admin", "uuid": "e423f3e7-16f6-11ec-a2d3-1e00b900043d"}' is allowed to perform API calls: 0.0.0.0/0,::/0
2021-09-17 11:26:32,669 DEBUG [c.c.a.ApiServlet] (qtp769798433-15:ctx-9d0ad7da ctx-89672168) (logid:30c456f4) ===END===  172.16.250.3 -- GET  jobId=f29d9a58-fd07-4186-bc85-ea2cc60ac1f3&command=queryAsyncJobResult&response=json
2021-09-17 11:26:32,670 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) Updating resource Type = volume count for Account = 2 Operation = decreasing Amount = 1
2021-09-17 11:26:32,673 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) Updating resource Type = primary_storage count for Account = 2 Operation = decreasing Amount = (50.00 MB) 52428800
2021-09-17 11:26:32,680 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) alertType=[8] dataCenterId=[1] podId=[null] clusterId=[null] message=[Failed to deploy Vm with Id: 7, on Host with Id: null].
2021-09-17 11:26:32,683 WARN  [c.c.a.AlertManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) No recipients set in 'alert.email.addresses', skipping sending alert with subject: Failed to deploy Vm with Id: 7, on Host with Id: null and content: Failed to deploy Vm with Id: 7, on Host with Id: null
2021-09-17 11:26:32,685 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) Updating resource Type = user_vm count for Account = 2 Operation = decreasing Amount = 1
2021-09-17 11:26:32,689 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) Updating resource Type = cpu count for Account = 2 Operation = decreasing Amount = 1
2021-09-17 11:26:32,692 DEBUG [c.c.r.ResourceLimitManagerImpl] (API-Job-Executor-47:ctx-5239873f job-69 ctx-448d1fc4) (logid:f29d9a58) Updating resource Type = memory count for Account = 2 Operation = decreasing Amount = 512
2021-09-17 11:26:32,699 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-47:ctx-5239873f job-69) (logid:f29d9a58) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
com.cloud.utils.exception.CloudRuntimeException: No suitable host found for follow compute offering tags: 
        at com.cloud.host.dao.HostDaoImpl.findHostByComputeOfferings(HostDaoImpl.java:1255)
        at com.cloud.host.dao.HostDaoImpl.listByHostTag(HostDaoImpl.java:770)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at com.sun.proxy.$Proxy46.listByHostTag(Unknown Source)
        at com.cloud.agent.manager.allocator.impl.FirstFitAllocator.allocateTo(FirstFitAllocator.java:177)
        at com.cloud.agent.manager.allocator.impl.FirstFitAllocator.allocateTo(FirstFitAllocator.java:106)
        at com.cloud.agent.manager.allocator.impl.FirstFitRoutingAllocator.allocateTo(FirstFitRoutingAllocator.java:41)
        at com.cloud.deploy.DeploymentPlanningManagerImpl.findSuitableHosts(DeploymentPlanningManagerImpl.java:1541)
        at com.cloud.deploy.DeploymentPlanningManagerImpl.checkClustersforDestination(DeploymentPlanningManagerImpl.java:1212)
        at com.cloud.deploy.DeploymentPlanningManagerImpl.planDeployment(DeploymentPlanningManagerImpl.java:539)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.reserveVirtualMachine(VMEntityManagerImpl.java:203)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.reserve(VirtualMachineEntityImpl.java:202)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:5090)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4617)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4606)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

not sure if these are new, but I see a number of

021-09-17 11:24:09,054 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-43:ctx-e44070b0 job-65) (logid:97bfaefa) Complete async job-65, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"431","errortext":"Invalid resource type: Host"}

as well

i'll update with more logs if i find out more.

DaanHoogland commented Sep 17, 2021

Copy link
Copy Markdown
Contributor

follow up:
when I remove the tag from the host the template still won't deploy.
setting the tag "none" onboth the template and one of the hosts, the deploy proceeds, and a VM ends out on the tagged host.
@ravening, can you look at my scenario, please?

ravening commented Sep 20, 2021

Copy link
Copy Markdown
Member Author

follow up:
when I remove the tag from the host the template still won't deploy.
setting the tag "none" onboth the template and one of the hosts, the deploy proceeds, and a VM ends out on the tagged host.
@ravening, can you look at my scenario, please?

@DaanHoogland i will look into it and get back to you

nvazquez modified the milestones: 4.16.0.0, 4.17.0.0 Sep 26, 2021
DaanHoogland assigned ravening and unassigned DaanHoogland Oct 4, 2021

nvazquez commented Feb 18, 2022

Copy link
Copy Markdown
Contributor

Hi @ravening can you please fix the conflicts?

nvazquez modified the milestones: 4.17.0.0, 4.18.0.0 Mar 6, 2022

github-actions Bot commented Apr 7, 2022

Copy link
Copy Markdown

Hi @${author}, your pull request has merge conflicts. Can you fix the conflicts and sync your branch with the base branch?

ravening and others added 3 commits April 11, 2022 16:07
Match template tags with host tag for vm deployment
f846c0e 
1. Match the template tags with host tags for vm deployment/migration
2. Allow updating templates/isos with template tag by admin
3. Allow adding template tag while registering/creating template/iso
4. Allow searching templats/isos using template tags

If a template has template tag then deploy the vm which uses this
template on hosts which matches the template tags. If no suitable
hosts are found then throw exception

Similarly match the template tag with host tags while starting or
migrating a vm to a different host
Add test case for listing template with tags
c62bfce
fix build error
73c3792

ravening commented Apr 11, 2022

Copy link
Copy Markdown
Member Author

@nvazquez done
@weizhouapache please review it

github-actions Bot commented Apr 22, 2022

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

ravening closed this Jun 29, 2022
DaanHoogland removed this from the 4.18.0.0 milestone Oct 5, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

DaanHoogland DaanHoogland requested changes
nvazquez nvazquez approved these changes
weizhouapache weizhouapache approved these changes

Assignees

ravening

Labels

component:api component:integration-test component:marvin component:UI Python Warning... Python code Ahead! status:has-conflicts type:enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

Back | FazBrowse Home | New Git URL