server: check server capacity when start/deploy a vm by weizhouapache · Pull Request #5339 · apache/cloudstack

weizhouapache · 2021-08-19T09:00:05Z

Description

This PR fixes the issue that server capacity check is skipped, when start or migrate a vm with specified hostid.

=======UPDATE 2021-09-01===========
We have decided to only add capacity check. the other checks (host state, affinity group, etc) are not added.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

weizhouapache · 2021-08-19T09:11:47Z

@blueorangutan package

blueorangutan · 2021-08-19T09:12:03Z

@weizhouapache a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

weizhouapache · 2021-08-19T09:24:11Z

@andrijapanicsb @nvazquez

the following checks are added when deploy/start a vm with specified id.
(1) host state (Up and Enabled)
(2) DPDK enabled (if it is enabled on vm, host should be enabled also)
(3) hypervisor capability (max guest limits)
(4) server capacity

not sure if users are happy with the checks.

ps: (1) to (3) already exist in destination host check in vm migration. (4) is a new check in vm migration.

=======UPDATE 2021-09-01===========

We have decided to only add capacity check. the other checks are not added.

blueorangutan · 2021-08-19T09:38:24Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 932

weizhouapache · 2021-08-19T09:56:24Z

@blueorangutan test

blueorangutan · 2021-08-19T09:57:03Z

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-08-19T20:01:02Z

Trillian test result (tid-1721)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34502 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5339-t1721-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

nvazquez

Thanks @weizhouapache looks good. What about host tags checks against the offering? cc. @andrijapanicsb

sureshanaparti

clgtm, minor changes to code format required (check suggestions)

weizhouapache · 2021-08-23T12:41:29Z

@nvazquez
actually vm cannot be started or migrated if there is no enough server capacity in 4.15 and master.

the process to deploy/start a vm without destination host is
(1) check server capacity of all hosts
(2) select a host
(3) double check if host has enough capacity (because step 1 might take long time if there are many hosts). this step was added by #3728

if destination host is specified, step 1 and 2 are skipped for now. but there is still a new check in step 3.
I am not sure if necessary to add an extra step to check server capacity (it takes few milliseconds) in this PR.

current error

allow.deploy.vm.if.deploy.on.given.host.fails=false
allow.deploy.vm.if.deploy.on.given.host.fails=true

weizhouapache · 2021-08-23T12:42:19Z

clgtm, minor changes to code format required (check suggestions)

thanks @sureshanaparti , changes have been committed.

GutoVeronezi · 2021-08-23T17:58:49Z

+                    deployOnGivenHost = true;
+                }
+            } catch (InvalidParameterValueException ex) {
+                s_logger.error("Cannot deploy the VM to specified host due to: " + ex.getMessage());


We could pass the exception to the log.

davidjumani · 2021-08-24T05:12:47Z

-            if (!AllowDeployVmIfGivenHostFails.value()) {
-                deployOnGivenHost = true;
+            try {
+                checkVmDestinationServerCapacity(vm, destinationHost);


Would it be simpler to use _capacityMgr.checkIfHostHasCpuCapabilityAndCapacity ?

@davidjumani I will change it.thanks

weizhouapache · 2021-08-24T08:19:23Z

@nvazquez @davidjumani updated this pr.

allow.deploy.vm.if.deploy.on.given.host.fails=false

allow.deploy.vm.if.deploy.on.given.host.fails=true

yadvr · 2021-08-27T10:53:52Z

@weizhouapache does this affect 4.15 too? If so, pl raise PR against 4.15 branch.

weizhouapache · 2021-08-27T11:18:55Z

@weizhouapache does this affect 4.15 too? If so, pl raise PR against 4.15 branch.

@rhtyd @nvazquez changed base branch to 4.15

nvazquez · 2021-08-30T23:39:08Z

@blueorangutan package

blueorangutan · 2021-08-30T23:40:03Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-08-31T00:17:36Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 1065

nvazquez · 2021-08-31T01:49:07Z

@blueorangutan test ubuntu18 kvm-ubuntu18

blueorangutan · 2021-08-31T01:50:02Z

@nvazquez a Trillian-Jenkins test job (ubuntu18 mgmt + kvm-ubuntu18) has been kicked to run smoke tests

blueorangutan · 2021-08-31T15:24:40Z

Trillian test result (tid-1845)
Environment: kvm-ubuntu18 (x2), Advanced Networking with Mgmt server u18
Total time taken: 47260 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5339-t1845-kvm-ubuntu18.zip
Intermittent failure detected: /marvin/tests/smoke/test_accounts.py
Intermittent failure detected: /marvin/tests/smoke/test_deploy_vm_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_domain_vpc_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 83 look OK, 4 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestAccounts>:setup	`Error`	0.00	test_accounts.py
ContextSuite context=TestAddVmToSubDomain>:setup	`Error`	0.00	test_accounts.py
test_DeleteDomain	`Error`	0.89	test_accounts.py
test_forceDeleteDomain	`Failure`	0.88	test_accounts.py
test_forceDeleteDomain	`Error`	3.10	test_accounts.py
ContextSuite context=TestRemoveUserFromAccount>:setup	`Error`	5.39	test_accounts.py
ContextSuite context=TestTemplateHierarchy>:teardown	`Error`	425.51	test_accounts.py
test_03_1_create_iso_with_checksum_md5_negative	`Error`	999.88	test_iso.py
test_03_create_iso_with_checksum_md5	`Error`	65.47	test_iso.py
test_01_migrate_VM_and_root_volume	`Error`	72.08	test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks	`Error`	51.01	test_vm_life_cycle.py
test_01_secure_vm_migration	`Error`	112.08	test_vm_life_cycle.py
test_02_unsecure_vm_migration	`Error`	170.18	test_vm_life_cycle.py
test_08_migrate_vm	`Error`	48.24	test_vm_life_cycle.py
test_hostha_kvm_host_degraded	`Error`	681.86	test_hostha_kvm.py
test_hostha_kvm_host_fencing	`Error`	666.57	test_hostha_kvm.py

nvazquez · 2021-08-31T16:38:59Z

@blueorangutan test

blueorangutan · 2021-08-31T16:40:03Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-09-01T07:46:14Z

Trillian test result (tid-1869)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 52093 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5339-t1869-kvm-centos7.zip
Smoke tests completed. 88 look OK, 3 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_vm_start_failure	`Error`	63.18	test_deploy_vm.py
test_deploy_vm_volume_creation_failure	`Error`	66.22	test_deploy_vm.py
test_vm_ha	`Error`	60.51	test_vm_ha.py
test_vm_sync	`Error`	125.86	test_vm_sync.py

yadvr · 2021-09-02T05:31:33Z

Lgtm, failures not related to this PR.
@weizhouapache @nvazquez @borisstoyanov @vladimirpetrov anybody testing this?

nvazquez · 2021-09-02T13:51:14Z

Kicking a new round of tests while is being tested
@blueorangutan test

blueorangutan · 2021-09-02T13:53:06Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

vladimirpetrov

LGTM based on manual testing. Tested scenarios:

(1) cpu speed must be smaller than or equals to cpu mhz of physical host (this does not consider over provisioning)
(2) VM with more cores than available on the host and enough CPU speed and memory
(3) host must have enough cpu resource (vm cpu*speed)
(4) host must have enough memory

blueorangutan · 2021-09-02T23:55:56Z

Trillian test result (tid-1943)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34076 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5339-t1943-kvm-centos7.zip
Smoke tests completed. 87 look OK, 0 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File

yadvr added this to the 4.16.0.0 milestone Aug 19, 2021

nvazquez reviewed Aug 20, 2021

View reviewed changes

nvazquez closed this Aug 20, 2021

nvazquez reopened this Aug 20, 2021

sureshanaparti reviewed Aug 23, 2021

View reviewed changes

Comment thread server/src/main/java/com/cloud/vm/UserVmManagerImpl.java Outdated

sureshanaparti reviewed Aug 23, 2021

View reviewed changes

Comment thread server/src/main/java/com/cloud/vm/UserVmManagerImpl.java Outdated

sureshanaparti approved these changes Aug 23, 2021

View reviewed changes

GutoVeronezi reviewed Aug 23, 2021

View reviewed changes

davidjumani reviewed Aug 24, 2021

View reviewed changes

weizhouapache closed this Aug 24, 2021

weizhouapache reopened this Aug 24, 2021

davidjumani approved these changes Aug 25, 2021

View reviewed changes

weizhouapache changed the title ~~server: check server capacity when start/deploy or migrate a vm~~ server: check server capacity when start/deploy migrate a vm Aug 25, 2021

weizhouapache changed the title ~~server: check server capacity when start/deploy migrate a vm~~ server: check server capacity when start/deploy a vm Aug 25, 2021

weizhouapache closed this Aug 25, 2021

weizhouapache reopened this Aug 25, 2021

weizhouapache closed this Aug 26, 2021

weizhouapache reopened this Aug 26, 2021

weizhouapache added the status:needs-testing label Aug 26, 2021

server: check server capacity when start/deploy a vm

de39782

weizhouapache force-pushed the 4.16-check-server-capacity-start-or-migrate-vm branch from 0e47334 to de39782 Compare August 27, 2021 11:14

weizhouapache changed the base branch from main to 4.15 August 27, 2021 11:18

apache deleted a comment from blueorangutan Aug 31, 2021

yadvr modified the milestones: 4.16.0.0, 4.15.2.0 Aug 31, 2021

weizhouapache mentioned this pull request Sep 1, 2021

server: fix reset sshkey is broken in master/4.16 #5390

Merged

12 tasks

vladimirpetrov approved these changes Sep 2, 2021

View reviewed changes

nvazquez approved these changes Sep 3, 2021

View reviewed changes

nvazquez merged commit cf6dc66 into apache:4.15 Sep 3, 2021

weizhouapache deleted the 4.16-check-server-capacity-start-or-migrate-vm branch December 9, 2022 08:45

DaanHoogland removed the status:needs-testing label Jan 28, 2026

Conversation

weizhouapache commented Aug 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

weizhouapache commented Aug 19, 2021

Uh oh!

blueorangutan commented Aug 19, 2021

Uh oh!

weizhouapache commented Aug 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Aug 19, 2021

Uh oh!

weizhouapache commented Aug 19, 2021

Uh oh!

blueorangutan commented Aug 19, 2021

Uh oh!

blueorangutan commented Aug 19, 2021

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Aug 23, 2021

Uh oh!

weizhouapache commented Aug 23, 2021

Uh oh!

GutoVeronezi Aug 23, 2021

Choose a reason for hiding this comment

Uh oh!

davidjumani Aug 24, 2021

Choose a reason for hiding this comment

Uh oh!

weizhouapache Aug 24, 2021

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Aug 24, 2021

Uh oh!

yadvr commented Aug 27, 2021

Uh oh!

weizhouapache commented Aug 27, 2021

Uh oh!

nvazquez commented Aug 30, 2021

Uh oh!

blueorangutan commented Aug 30, 2021

Uh oh!

blueorangutan commented Aug 31, 2021

Uh oh!

nvazquez commented Aug 31, 2021

Uh oh!

blueorangutan commented Aug 31, 2021

Uh oh!

blueorangutan commented Aug 31, 2021

Uh oh!

nvazquez commented Aug 31, 2021

Uh oh!

blueorangutan commented Aug 31, 2021

Uh oh!

blueorangutan commented Sep 1, 2021

Uh oh!

yadvr commented Sep 2, 2021

Uh oh!

nvazquez commented Sep 2, 2021

Uh oh!

blueorangutan commented Sep 2, 2021

Uh oh!

vladimirpetrov left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Aug 19, 2021 •

edited

Loading

weizhouapache commented Aug 19, 2021 •

edited

Loading