Skip to content

Fixed incorrect error message on invalid template type download#4138

Merged
yadvr merged 7 commits into
apache:4.13from
shapeblue:error_message_on_template_type_mismatch
Jul 15, 2020
Merged

Fixed incorrect error message on invalid template type download#4138
yadvr merged 7 commits into
apache:4.13from
shapeblue:error_message_on_template_type_mismatch

Conversation

@Spaceman1984
Copy link
Copy Markdown
Contributor

@Spaceman1984 Spaceman1984 commented Jun 10, 2020

Description

When a template is downloaded, the first 1MB of the template is validated to determine if the template is of correct file type. On failure, the download is aborted and the input stream is set to null. This leads to a second error when the try-with-resources block tries to auto-close the stream and throws an ioexception. The "stream closed" error message is then written to the db.

This PR checks if an error has been stored before setting a new error message.

Fixes: #4127

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):

How Has This Been Tested?

This has been tested by downloading an invalid template and checking the error message from the UI or in cloud.template_store_ref.error_str in the db.
The cleanup has been tested by restarting the management server to trigger template sync to verify if the dead template has been removed.

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1330

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Comment thread core/src/main/java/com/cloud/storage/template/HttpTemplateDownloader.java Outdated
@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1333

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Comment thread core/src/main/java/com/cloud/storage/template/HttpTemplateDownloader.java Outdated
@Spaceman1984 Spaceman1984 marked this pull request as draft June 22, 2020 13:59
@Spaceman1984 Spaceman1984 marked this pull request as ready for review June 23, 2020 09:18
@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1445

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@nvazquez nvazquez marked this pull request as ready for review July 10, 2020 15:20
@nvazquez
Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1564

@nvazquez
Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link
Copy Markdown

Trillian test result (tid-2055)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 59213 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4138-t2055-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_public_ip_range.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_secondary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 55 look OK, 22 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_02_vpc_privategw_static_routes Failure 171.90 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 170.45 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Failure 223.65 test_privategw_acl.py
ContextSuite context=TestResetVmOnReboot>:setup Error 0.00 test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup Error 0.00 test_resource_accounting.py
ContextSuite context=TestRouterDHCPHosts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup Error 0.00 test_router_dns.py
test_01_sys_vm_start Failure 0.07 test_secondary_storage.py
ContextSuite context=TestRouterDnsService>:setup Error 0.00 test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup Error 0.00 test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup Error 0.00 test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup Error 0.14 test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup Error 0.00 test_snapshots.py
test_01_list_sec_storage_vm Failure 0.03 test_ssvm.py
test_02_list_cpvm_vm Failure 0.02 test_ssvm.py
test_03_ssvm_internals Failure 0.02 test_ssvm.py
test_04_cpvm_internals Failure 0.03 test_ssvm.py
test_05_stop_ssvm Failure 0.02 test_ssvm.py
test_06_stop_cpvm Failure 0.02 test_ssvm.py
test_07_reboot_ssvm Failure 0.02 test_ssvm.py
test_08_reboot_cpvm Failure 0.02 test_ssvm.py
test_09_destroy_ssvm Failure 0.02 test_ssvm.py
test_10_destroy_cpvm Failure 0.02 test_ssvm.py
test_02_create_template_with_checksum_sha1 Error 65.36 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.37 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.36 test_templates.py
test_05_create_template_with_no_checksum Error 65.36 test_templates.py
test_02_deploy_vm_from_direct_download_template Error 1.16 test_templates.py
test_03_deploy_vm_wrong_checksum Error 1.22 test_templates.py
ContextSuite context=TestTemplates>:setup Error 16.49 test_templates.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestLBRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestNatRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestPublicIPUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestSnapshotUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVmUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVolumeUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVpnUsage>:setup Error 0.00 test_usage.py
ContextSuite context=Test01DeployVM>:setup Error 0.00 test_vm_life_cycle.py
ContextSuite context=Test02VMLifeCycle>:setup Error 0.00 test_vm_life_cycle.py
test_14_secure_to_secure_vm_migration Error 11.26 test_vm_life_cycle.py
test_15_secured_to_nonsecured_vm_migration Error 73.92 test_vm_life_cycle.py
test_16_nonsecured_to_secured_vm_migration Error 1.17 test_vm_life_cycle.py
ContextSuite context=TestVmSnapshot>:setup Error 1.56 test_vm_snapshots.py
ContextSuite context=TestCreateVolume>:setup Error 0.00 test_volumes.py
ContextSuite context=TestVolumes>:setup Error 0.00 test_volumes.py
ContextSuite context=TestVPCRedundancy>:setup Error 0.00 test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup Error 0.00 test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
test_disable_oobm_ha_state_ineligible Error 1511.72 test_hostha_kvm.py

@nvazquez
Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1567

@nvazquez
Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link
Copy Markdown

Trillian test result (tid-2063)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 59961 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4138-t2063-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_public_ip_range.py
Intermittent failure detected: /marvin/tests/smoke/test_reset_vm_on_reboot.py
Intermittent failure detected: /marvin/tests/smoke/test_resource_accounting.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dns.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dnsservice.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_iptables_default_policy.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_routers.py
Intermittent failure detected: /marvin/tests/smoke/test_secondary_storage.py
Intermittent failure detected: /marvin/tests/smoke/test_service_offerings.py
Intermittent failure detected: /marvin/tests/smoke/test_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_ssvm.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_snapshots.py
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 55 look OK, 22 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_02_vpc_privategw_static_routes Failure 181.47 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 176.01 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Failure 240.47 test_privategw_acl.py
ContextSuite context=TestResetVmOnReboot>:setup Error 0.00 test_reset_vm_on_reboot.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup Error 0.00 test_resource_accounting.py
ContextSuite context=TestRouterDHCPHosts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup Error 0.00 test_router_dns.py
test_01_sys_vm_start Failure 0.08 test_secondary_storage.py
ContextSuite context=TestRouterDnsService>:setup Error 0.00 test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup Error 0.00 test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup Error 0.00 test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup Error 0.14 test_service_offerings.py
ContextSuite context=TestSnapshotRootDisk>:setup Error 0.00 test_snapshots.py
test_01_list_sec_storage_vm Failure 0.03 test_ssvm.py
test_02_list_cpvm_vm Failure 0.02 test_ssvm.py
test_03_ssvm_internals Failure 0.02 test_ssvm.py
test_04_cpvm_internals Failure 0.02 test_ssvm.py
test_05_stop_ssvm Failure 0.02 test_ssvm.py
test_06_stop_cpvm Failure 0.02 test_ssvm.py
test_07_reboot_ssvm Failure 0.03 test_ssvm.py
test_08_reboot_cpvm Failure 0.02 test_ssvm.py
test_09_destroy_ssvm Failure 0.02 test_ssvm.py
test_10_destroy_cpvm Failure 0.02 test_ssvm.py
test_02_create_template_with_checksum_sha1 Error 65.36 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.38 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.35 test_templates.py
test_05_create_template_with_no_checksum Error 65.36 test_templates.py
test_02_deploy_vm_from_direct_download_template Error 1.19 test_templates.py
test_03_deploy_vm_wrong_checksum Error 1.24 test_templates.py
ContextSuite context=TestTemplates>:setup Error 16.55 test_templates.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestLBRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestNatRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestPublicIPUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestSnapshotUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVmUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVolumeUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVpnUsage>:setup Error 0.00 test_usage.py
ContextSuite context=Test01DeployVM>:setup Error 0.00 test_vm_life_cycle.py
ContextSuite context=Test02VMLifeCycle>:setup Error 0.00 test_vm_life_cycle.py
test_14_secure_to_secure_vm_migration Error 11.27 test_vm_life_cycle.py
test_15_secured_to_nonsecured_vm_migration Error 73.93 test_vm_life_cycle.py
test_16_nonsecured_to_secured_vm_migration Error 1.18 test_vm_life_cycle.py
ContextSuite context=TestVmSnapshot>:setup Error 1.55 test_vm_snapshots.py
ContextSuite context=TestCreateVolume>:setup Error 0.00 test_volumes.py
ContextSuite context=TestVolumes>:setup Error 0.00 test_volumes.py
ContextSuite context=TestVPCRedundancy>:setup Error 0.00 test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup Error 0.00 test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
test_disable_oobm_ha_state_ineligible Error 1512.18 test_hostha_kvm.py

@nvazquez
Copy link
Copy Markdown
Contributor

@Spaceman1984 can you take a look at the failing tests?

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@nvazquez I had a look at the errors, doesn't seem related to template downloads. I will fire the tests again.

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos7 ✔debian. JID-1569

@Spaceman1984
Copy link
Copy Markdown
Contributor Author

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@Spaceman1984 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link
Copy Markdown

Trillian test result (tid-2097)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28268 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4138-t2097-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Smoke tests completed. 76 look OK, 1 have error(s)
Only failed tests results shown below:

Test Result Time (s) Test File
test_02_vpc_privategw_static_routes Failure 182.40 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 181.51 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Failure 232.38 test_privategw_acl.py

@yadvr yadvr added this to the 4.13.2.0 milestone Jul 15, 2020
@yadvr yadvr merged commit affd010 into apache:4.13 Jul 15, 2020
shwstppr pushed a commit to shapeblue/cloudstack that referenced this pull request Jul 20, 2020
…nload (apache#4138)

When a template is downloaded, the first 1MB of the template is validated to determine if the template is of correct file type. On failure, the download is aborted and the input stream is set to null. This leads to a second error when the try-with-resources block tries to auto-close the stream and throws an ioexception. The "stream closed" error message is then written to the db.

This PR checks if an error has been stored before setting a new error message.

Fixes: apache#4127
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants