FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Flexible timeout for commands MS sends to the agents by GutoVeronezi · Pull Request #8605 · apache/cloudstack · GitHub

  1. apache
  2. cloudstack

Repository navigation

Flexible timeout for commands MS sends to the agents - #8605

Closed
GutoVeronezi wants to merge 29 commits into
apache:mainfrom
GutoVeronezi:flexible-command-timeout
ConversationCommits (29)ChecksFiles changed
Closed

Flexible timeout for commands MS sends to the agents #8605
GutoVeronezi wants to merge 29 commits into
apache:mainfrom
GutoVeronezi:flexible-command-timeout

Conversation

GutoVeronezi commented Feb 3, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

Currently, almost every command MS sends to the agents uses the global setting wait as the timeout's value. That means, if you want to extend or shorten the execution of a command, you will affect every other command. Therefore, currently we do not have a way to customize the timeout for specific commands.

For some edge cases, we have "wait"/"timeout" hardcoded values or values that come from global settings to set the commands timeout, like for GetAutoScaleMetricsCommand and for MigrateCommand. Currently, we have 385 classes that extend com.cloud.agent.api.Command (I skipped classes that extend com.cloud.agent.api.Answer); therefore, externalizing global settings for each command to provide flexibility is not feasible; however, we can reach flexibility by other means.

Currently, if we do not set a wait to the command, AgentManagerImpl will fallback to the wait global setting:

cloudstack/engine/orchestration/src/main/java/com/cloud/agent/manager/AgentManagerImpl.java

Lines 441 to 443 in af8a582

if (timeout <= 0) {
timeout = Wait.value();
}

I propose to create a table of command_timeout to store the classpath of the command and its wait/timeout. Then, instead of fall backing directly to the wait global setting, we check if there is an entry of the command in this table. If it exists, then we use the entry's value; otherwise, we fall back to the wait global setting (the current behavior). Thus, we can change the commands wait/timeout on demand and in a flexible way.

The first step of the proposal is to create the table and changing AgentManagerImpl to check the table. Some entries will already be created for hardcoded values. Further steps can be found in #8506.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

How Has This Been Tested?

TBD

GutoVeronezi marked this pull request as draft February 3, 2024 00:11

codecov Bot commented Feb 3, 2024 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 37.50000% with 25 lines in your changes missing coverage. Please review.

Project coverage is 15.28%. Comparing base (19f9635) to head (6d44719).
Report is 566 commits behind head on main.

Files with missing lines Patch % Lines
...ck/framework/config/dao/CommandTimeoutDaoImpl.java 0.00% 13 Missing ⚠️
...dstack/framework/config/impl/CommandTimeoutVO.java 0.00% 7 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java 78.94% 4 Missing ⚠️
...rc/main/java/com/cloud/agent/manager/Commands.java 0.00% 1 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##               main    #8605     +/-   ##
===========================================
  Coverage     15.28%   15.28%             
- Complexity    11535    11538      +3     
===========================================
  Files          5424     5426      +2     
  Lines        474334   474361     +27     
  Branches      60970    58437   -2533     
===========================================
+ Hits          72498    72503      +5     
- Misses       393778   393804     +26     
+ Partials       8058     8054      -4
Flag Coverage Δ
uitests 4.25% <ø> (ø)
unittests 16.02% <37.50%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

GutoVeronezi mentioned this pull request Feb 3, 2024
Closed

github-actions Bot commented Feb 5, 2024

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland reviewed Feb 6, 2024
View reviewed changes

DaanHoogland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

i think this does what you proposed (clgtm) but we have to think about versions and upgrade paths

Comment thread engine/orchestration/src/main/java/com/cloud/agent/manager/AgentManagerImpl.java Resolved
Comment thread engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Resolved
Comment thread engine/schema/src/main/resources/META-INF/db/schema-41900to41910.sql Resolved

andrijapanicsb commented Mar 29, 2024

Copy link
Copy Markdown
Contributor

@GutoVeronezi thx for the detailed explanation on this one, I guess we'll have to go with that - I'm now more clear on the idea, and, indeed, it doesn't align with what I was "expecting" to be able to do as on operator, that would have to be a separate effort/PR

github-actions Bot commented May 30, 2024

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

@GutoVeronezi do we need more work on this? (except for conflict resolution)

GutoVeronezi marked this pull request as ready for review June 4, 2024 15:10

GutoVeronezi commented Jun 4, 2024

Copy link
Copy Markdown
Contributor Author

@GutoVeronezi do we need more work on this? (except for conflict resolution)

I think we are good with the code; however, I want to try it a bit more and post the tests.

DaanHoogland reopened this Jun 4, 2024
DaanHoogland reopened this Jun 4, 2024

DaanHoogland commented Jun 5, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan package

blueorangutan commented Jun 5, 2024

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan commented Jun 5, 2024

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9787

DaanHoogland commented Jun 6, 2024

Copy link
Copy Markdown
Contributor

@blueorangutan test

blueorangutan commented Jun 6, 2024

Copy link
Copy Markdown

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan commented Jun 6, 2024

Copy link
Copy Markdown

[SF] Trillian test result (tid-10370)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49632 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8605-t10370-kvm-centos7.zip
Smoke tests completed. 132 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

github-actions Bot commented Jun 10, 2024

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland linked an issue Jun 18, 2024 that may be closed by this pull request
improvement: externalize timeout for hypervisor/agent commands #8506
Closed
JoaoJandre modified the milestones: 4.20.0.0, 4.21.0.0 Sep 10, 2024
GutoVeronezi mentioned this pull request Dec 3, 2024
Merged
2 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

DaanHoogland DaanHoogland approved these changes
vishesh92 Awaiting requested review from vishesh92
vladimirpetrov Awaiting requested review from vladimirpetrov

Assignees

No one assigned

Labels

component:agent component:api component:networking component:orchestration component:storage status:has-conflicts

Projects

No open projects
Apache CloudStack 4.21.0
Status: Done

Milestone

4.21.0

Development

Successfully merging this pull request may close these issues.

improvement: externalize timeout for hypervisor/agent commands

5 participants

Back | FazBrowse Home | New Git URL