Ova download fix by DaanHoogland · Pull Request #6758 · apache/cloudstack

DaanHoogland · 2022-09-22T09:24:29Z

Description

This PR allows for downloading OVA with the read properties from OVF flag when the OVF does not contain a minimum hardware version.

Fixes: #6736

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

by uploading an OVA that does not provide a minimum hardware version and making sure it is accepted.

DaanHoogland · 2022-09-22T09:25:07Z

@blueorangutan package

blueorangutan · 2022-09-22T09:26:03Z

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

acs-robot · 2022-09-22T09:26:10Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-09-22T09:27:04Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-09-22T09:33:59Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6758 (SL-JID-2403)

blueorangutan · 2022-09-22T10:03:46Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 4245

codecov · 2022-09-22T11:29:33Z

Codecov Report

Merging #6758 (37b5548) into main (65c7070) will increase coverage by 4.73%.
The diff coverage is 0.00%.

❗ Current head 37b5548 differs from pull request most recent head 77ec01c. Consider uploading reports for the commit 77ec01c to get more accurate results

@@             Coverage Diff              @@
##               main    #6758      +/-   ##
============================================
+ Coverage      5.88%   10.62%   +4.73%     
- Complexity     3942     6869    +2927     
============================================
  Files          2454     2466      +12     
  Lines        242723   244565    +1842     
  Branches      37988    38277     +289     
============================================
+ Hits          14289    25974   +11685     
+ Misses       226850   215309   -11541     
- Partials       1584     3282    +1698

Impacted Files	Coverage Δ
...om/cloud/storage/dao/GuestOSHypervisorDaoImpl.java	`32.25% <0.00%> (-0.36%)`	⬇️
...storage/image/deployasis/DeployAsIsHelperImpl.java	`13.26% <0.00%> (-0.21%)`	⬇️
...rce/wrapper/LibvirtResizeVolumeCommandWrapper.java	`49.50% <0.00%> (-27.17%)`	⬇️
.../cloud/hypervisor/kvm/storage/KVMPhysicalDisk.java	`70.27% <0.00%> (-10.98%)`	⬇️
...pper/LibvirtPrepareForMigrationCommandWrapper.java	`43.10% <0.00%> (-7.97%)`	⬇️
...rg/apache/cloudstack/quota/QuotaStatementImpl.java	`36.28% <0.00%> (-3.99%)`	⬇️
...loud/hypervisor/kvm/resource/LibvirtSecretDef.java	`60.00% <0.00%> (-3.16%)`	⬇️
.../hypervisor/kvm/storage/ScaleIOStorageAdaptor.java	`10.48% <0.00%> (-2.63%)`	⬇️
...apache/cloudstack/storage/volume/VolumeObject.java	`35.75% <0.00%> (-2.61%)`	⬇️
...vm/resource/wrapper/LibvirtStopCommandWrapper.java	`42.66% <0.00%> (-1.78%)`	⬇️
... and 539 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

DaanHoogland · 2022-09-26T09:47:51Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2022-09-26T09:49:03Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2022-09-26T13:20:05Z

Trillian test result (tid-5014)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 11465 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6758-t5014-vmware-67u3.zip
Smoke tests completed. 13 look OK, 1 have errors, 89 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_isolate_network_FW_PF_default_routes_egress_true	`Error`	981.10	test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false	`Error`	0.01	test_routers_network_ops.py
ContextSuite context=TestIsolatedNetworks>:teardown	`Error`	0.01	test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup	`Error`	0.03	test_routers_network_ops.py
all_test_accounts	`Skipped`	---	test_accounts.py
all_test_persistent_network	`Skipped`	---	test_persistent_network.py
all_test_affinity_groups	`Skipped`	---	test_affinity_groups.py
all_test_migration	`Skipped`	---	test_migration.py
all_test_affinity_groups_projects	`Skipped`	---	test_affinity_groups_projects.py
all_test_router_dnsservice	`Skipped`	---	test_router_dnsservice.py
all_test_annotations	`Skipped`	---	test_annotations.py
all_test_routers	`Skipped`	---	test_routers.py
all_test_async_job	`Skipped`	---	test_async_job.py
all_test_multipleips_per_nic	`Skipped`	---	test_multipleips_per_nic.py
all_test_attach_multiple_volumes	`Skipped`	---	test_attach_multiple_volumes.py
all_test_nested_virtualization	`Skipped`	---	test_nested_virtualization.py
all_test_backup_recovery_dummy	`Skipped`	---	test_backup_recovery_dummy.py
all_test_portable_publicip	`Skipped`	---	test_portable_publicip.py
all_test_certauthority_root	`Skipped`	---	test_certauthority_root.py
all_test_portforwardingrules	`Skipped`	---	test_portforwardingrules.py
all_test_console_endpoint	`Skipped`	---	test_console_endpoint.py
all_test_internal_lb	`Skipped`	---	test_internal_lb.py
all_test_network	`Skipped`	---	test_network.py
all_test_deploy_vgpu_enabled_vm	`Skipped`	---	test_deploy_vgpu_enabled_vm.py
all_test_network_acl	`Skipped`	---	test_network_acl.py
all_test_deploy_virtio_scsi_vm	`Skipped`	---	test_deploy_virtio_scsi_vm.py
all_test_privategw_acl	`Skipped`	---	test_privategw_acl.py
all_test_deploy_vm_iso	`Skipped`	---	test_deploy_vm_iso.py
all_test_privategw_acl_ovs_gre	`Skipped`	---	test_privategw_acl_ovs_gre.py
all_test_deploy_vm_iso_uefi	`Skipped`	---	test_deploy_vm_iso_uefi.py
all_test_network_permissions	`Skipped`	---	test_network_permissions.py
all_test_deploy_vm_root_resize	`Skipped`	---	test_deploy_vm_root_resize.py
all_test_nic	`Skipped`	---	test_nic.py
all_test_usage	`Skipped`	---	test_usage.py
all_test_deploy_vm_with_userdata	`Skipped`	---	test_deploy_vm_with_userdata.py
all_test_diagnostics	`Skipped`	---	test_diagnostics.py
all_test_projects	`Skipped`	---	test_projects.py
all_test_direct_download	`Skipped`	---	test_direct_download.py
all_test_public_ip_range	`Skipped`	---	test_public_ip_range.py
all_test_disk_offerings	`Skipped`	---	test_disk_offerings.py
all_test_non_contigiousvlan	`Skipped`	---	test_non_contigiousvlan.py
all_test_disk_provisioning_types	`Skipped`	---	test_disk_provisioning_types.py
all_test_outofbandmanagement	`Skipped`	---	test_outofbandmanagement.py
all_test_domain_disk_offerings	`Skipped`	---	test_domain_disk_offerings.py
all_test_outofbandmanagement_nestedplugin	`Skipped`	---	test_outofbandmanagement_nestedplugin.py
all_test_domain_network_offerings	`Skipped`	---	test_domain_network_offerings.py
all_test_over_provisioning	`Skipped`	---	test_over_provisioning.py
all_test_domain_service_offerings	`Skipped`	---	test_domain_service_offerings.py
all_test_pvlan	`Skipped`	---	test_pvlan.py
all_test_domain_vpc_offerings	`Skipped`	---	test_domain_vpc_offerings.py
all_test_scale_vm	`Skipped`	---	test_scale_vm.py
all_test_dynamicroles	`Skipped`	---	test_dynamicroles.py
all_test_ipv6_infra	`Skipped`	---	test_ipv6_infra.py
all_test_vm_life_cycle	`Skipped`	---	test_vm_life_cycle.py
all_test_enable_account_settings_for_domain	`Skipped`	---	test_enable_account_settings_for_domain.py
all_test_iso	`Skipped`	---	test_iso.py
all_test_secondary_storage	`Skipped`	---	test_secondary_storage.py
all_test_enable_role_based_users_in_projects	`Skipped`	---	test_enable_role_based_users_in_projects.py
all_test_regions	`Skipped`	---	test_regions.py
all_test_events_resource	`Skipped`	---	test_events_resource.py
all_test_password_server	`Skipped`	---	test_password_server.py
all_test_gateway_on_shared_networks	`Skipped`	---	test_gateway_on_shared_networks.py
all_test_reset_configuration_settings	`Skipped`	---	test_reset_configuration_settings.py
all_test_global_settings	`Skipped`	---	test_global_settings.py
all_test_reset_vm_on_reboot	`Skipped`	---	test_reset_vm_on_reboot.py
all_test_guest_vlan_range	`Skipped`	---	test_guest_vlan_range.py
all_test_resource_accounting	`Skipped`	---	test_resource_accounting.py
all_test_service_offerings	`Skipped`	---	test_service_offerings.py
all_test_resource_detail	`Skipped`	---	test_resource_detail.py
all_test_hostha_simulator	`Skipped`	---	test_hostha_simulator.py
all_test_human_readable_logs	`Skipped`	---	test_human_readable_logs.py
all_test_kubernetes_clusters	`Skipped`	---	test_kubernetes_clusters.py
all_test_login	`Skipped`	---	test_login.py
all_test_snapshots	`Skipped`	---	test_snapshots.py
all_test_kubernetes_supported_versions	`Skipped`	---	test_kubernetes_supported_versions.py
all_test_router_dhcphosts	`Skipped`	---	test_router_dhcphosts.py
all_test_list_ids_parameter	`Skipped`	---	test_list_ids_parameter.py
all_test_ssvm	`Skipped`	---	test_ssvm.py
all_test_loadbalance	`Skipped`	---	test_loadbalance.py
all_test_metrics_api	`Skipped`	---	test_metrics_api.py
all_test_vm_snapshots	`Skipped`	---	test_vm_snapshots.py
all_test_routers_iptables_default_policy	`Skipped`	---	test_routers_iptables_default_policy.py
all_test_storage_policy	`Skipped`	---	test_storage_policy.py
all_test_update_security_group	`Skipped`	---	test_update_security_group.py
all_test_vm_lifecycle_unmanage_import	`Skipped`	---	test_vm_lifecycle_unmanage_import.py
all_test_vm_snapshot_kvm	`Skipped`	---	test_vm_snapshot_kvm.py
all_test_volumes	`Skipped`	---	test_volumes.py
all_test_vpc_ipv6	`Skipped`	---	test_vpc_ipv6.py
all_test_vpc_redundant	`Skipped`	---	test_vpc_redundant.py
all_test_vpc_router_nics	`Skipped`	---	test_vpc_router_nics.py
all_test_vpc_vpn	`Skipped`	---	test_vpc_vpn.py
all_test_host_maintenance	`Skipped`	---	test_host_maintenance.py
all_test_hostha_kvm	`Skipped`	---	test_hostha_kvm.py

blueorangutan · 2022-09-27T03:43:20Z

Trillian test result (tid-5021)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43459 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6758-t5021-vmware-67u3.zip
Smoke tests completed. 102 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	764.42	test_kubernetes_clusters.py

DaanHoogland · 2022-09-27T07:36:38Z

@blueorangutan package

blueorangutan · 2022-09-27T07:37:03Z

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2022-09-27T08:34:04Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 4293

DaanHoogland · 2022-09-27T08:36:25Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2022-09-27T08:37:03Z

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2022-09-27T21:05:38Z

Trillian test result (tid-5031)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43658 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6758-t5031-vmware-67u3.zip
Smoke tests completed. 102 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	694.44	test_kubernetes_clusters.py

DaanHoogland · 2022-09-28T06:25:35Z

@borisstoyanov @nvazquez can you please look at this?

borisstoyanov · 2022-09-28T10:54:59Z

@blueorangutan test centos7 vmware-67u3 keepEnv

borisstoyanov · 2022-09-28T10:55:27Z

Sure @DaanHoogland, will do.

nvazquez

Code LGTM, didn't test it

…ge/deployasis/DeployAsIsHelperImpl.java

acs-robot · 2022-09-28T15:49:49Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-09-28T15:50:03Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-09-28T15:58:39Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6758 (SL-JID-2430)

borisstoyanov · 2022-09-29T13:19:18Z

@blueorangutan test centos7 vmware-67u3 keepEnv

blueorangutan · 2022-09-29T13:20:03Z

@borisstoyanov a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2022-09-30T01:56:44Z

Trillian test result (tid-5051)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44143 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6758-t5051-vmware-67u3.zip
Smoke tests completed. 102 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	`Failure`	627.03	test_kubernetes_clusters.py

borisstoyanov

Upload from local fails when the user does not select 'read from OVA'

acs-robot · 2022-10-13T09:39:58Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-10-13T09:41:03Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-10-13T09:53:37Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6758 (SL-JID-2493)

DaanHoogland · 2022-10-13T15:46:25Z

@blueorangutan package

blueorangutan · 2022-10-13T15:47:02Z

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

acs-robot · 2022-10-13T16:26:22Z

Found UI changes, kicking a new UI QA build
@blueorangutan ui

blueorangutan · 2022-10-13T16:27:03Z

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

blueorangutan · 2022-10-13T16:33:27Z

UI build: ✔️
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6758 (SL-JID-2495)

blueorangutan · 2022-10-13T16:34:49Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 4459

sonarqubecloud · 2022-10-13T17:28:59Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

DaanHoogland · 2022-10-14T14:24:44Z

Upload from local fails when the user does not select 'read from OVA'

@borisstoyanov I tested this with the macchinina template and it works, than I investigated the template for the cisco PI, it contains four disk definitions and only one vmdk. I am not sure if we should support this type of template, or how to handle it.

I also tested this on the latest 4.17 release (4.17.1.0) and it doesn´t work there either. I'd like to pull this out of scope for this PR.
Can you agree?

borisstoyanov

Just had a talk with @DaanHoogland and we agree that this template is not meant to be used without reading the properties, so it does not makes sense to look into that.
Also run a few upload tests and all seems to be good. LGTM

DaanHoogland added 2 commits September 20, 2022 16:37

log

ac0dce2

parentheses

3be72b9

boring-cyborg Bot added the component:storage label Sep 22, 2022

DaanHoogland requested review from borisstoyanov and nvazquez September 22, 2022 09:24

DaanHoogland mentioned this pull request Sep 23, 2022

OVA downloads into an error #6736

Closed

yadvr added this to the 4.18.0.0 milestone Sep 28, 2022

nvazquez approved these changes Sep 28, 2022

View reviewed changes

Comment thread ...orage/src/main/java/org/apache/cloudstack/storage/image/deployasis/DeployAsIsHelperImpl.java Outdated

Update engine/storage/src/main/java/org/apache/cloudstack/storage/ima…

1f13649

…ge/deployasis/DeployAsIsHelperImpl.java

borisstoyanov requested changes Sep 30, 2022

View reviewed changes

boring-cyborg Bot added component:secondary-storage component:ssvm labels Oct 13, 2022

npe investigative log

77ec01c

DaanHoogland force-pushed the ova-dd-error-fix branch from 37b5548 to 77ec01c Compare October 13, 2022 16:23

boring-cyborg Bot added the component:api label Oct 13, 2022

borisstoyanov approved these changes Oct 21, 2022

View reviewed changes

DaanHoogland marked this pull request as ready for review October 21, 2022 12:29

DaanHoogland merged commit 4a06363 into apache:main Oct 21, 2022

DaanHoogland mentioned this pull request Oct 21, 2022

default should map always, not require the minHypervisorVersion #6841

Closed

12 tasks

Conversation

DaanHoogland commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

DaanHoogland commented Sep 22, 2022

Uh oh!

blueorangutan commented Sep 22, 2022

Uh oh!

acs-robot commented Sep 22, 2022

Uh oh!

blueorangutan commented Sep 22, 2022

Uh oh!

blueorangutan commented Sep 22, 2022

Uh oh!

blueorangutan commented Sep 22, 2022

Uh oh!

codecov Bot commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

DaanHoogland commented Sep 26, 2022

Uh oh!

blueorangutan commented Sep 26, 2022

Uh oh!

blueorangutan commented Sep 26, 2022

Uh oh!

blueorangutan commented Sep 27, 2022

Uh oh!

DaanHoogland commented Sep 27, 2022

Uh oh!

blueorangutan commented Sep 27, 2022

Uh oh!

blueorangutan commented Sep 27, 2022

Uh oh!

DaanHoogland commented Sep 27, 2022

Uh oh!

blueorangutan commented Sep 27, 2022

Uh oh!

blueorangutan commented Sep 27, 2022

Uh oh!

DaanHoogland commented Sep 28, 2022

Uh oh!

borisstoyanov commented Sep 28, 2022

Uh oh!

borisstoyanov commented Sep 28, 2022

Uh oh!

nvazquez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

acs-robot commented Sep 28, 2022

Uh oh!

blueorangutan commented Sep 28, 2022

Uh oh!

blueorangutan commented Sep 28, 2022

Uh oh!

borisstoyanov commented Sep 29, 2022

Uh oh!

blueorangutan commented Sep 29, 2022

Uh oh!

blueorangutan commented Sep 30, 2022

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

acs-robot commented Oct 13, 2022

Uh oh!

blueorangutan commented Oct 13, 2022

Uh oh!

blueorangutan commented Oct 13, 2022

Uh oh!

DaanHoogland commented Oct 13, 2022

Uh oh!

DaanHoogland commented Sep 22, 2022 •

edited

Loading

codecov Bot commented Sep 22, 2022 •

edited

Loading

DaanHoogland commented Oct 14, 2022 •

edited

Loading