Log4j2 refactor cloud engine orchestration module by KlausDornsbach · Pull Request #8742 · apache/cloudstack

KlausDornsbach · 2024-03-04T14:13:09Z

Description

With the new version of Log4j there is some space for improving log readability using the new features. This PR refactors the logs from the cloud-orchestration module.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

How Has This Been Tested?

I have done some operations on cloudstack to induce the generation of the logs
No problems or changes to the log output were found

…odule

codecov · 2024-03-05T08:05:51Z

Codecov Report

Attention: Patch coverage is 27.91328% with 266 lines in your changes are missing coverage. Please review.

Project coverage is 30.80%. Comparing base (6dc3d06) to head (f323822).
Report is 1 commits behind head on main.

Files	Patch %	Lines
...cloud/agent/manager/ClusteredAgentManagerImpl.java	7.89%	105 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java	21.34%	70 Missing ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java	30.35%	39 Missing ⚠️
...ain/java/com/cloud/agent/manager/AgentAttache.java	31.81%	14 Missing and 1 partial ⚠️
...va/com/cloud/agent/manager/DirectAgentAttache.java	47.61%	11 Missing ⚠️
...com/cloud/agent/manager/ClusteredAgentAttache.java	0.00%	9 Missing ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java	83.33%	5 Missing ⚠️
...ack/engine/orchestration/DataMigrationUtility.java	0.00%	3 Missing ⚠️
...tack/engine/orchestration/StorageOrchestrator.java	0.00%	3 Missing ⚠️
...a/com/cloud/agent/manager/SynchronousListener.java	33.33%	1 Missing and 1 partial ⚠️
... and 3 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8742      +/-   ##
============================================
+ Coverage     30.69%   30.80%   +0.11%     
- Complexity    33463    33528      +65     
============================================
  Files          5397     5397              
  Lines        379397   379186     -211     
  Branches      55363    55175     -188     
============================================
+ Hits         116442   116825     +383     
+ Misses       247437   246935     -502     
+ Partials      15518    15426      -92

Flag	Coverage Δ
simulator-marvin-tests	`24.27% <27.37%> (+0.07%)`	⬆️
uitests	`4.34% <ø> (ø)`
unit-tests	`16.89% <1.08%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-03-08T10:41:05Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…odule

github-actions · 2024-03-13T13:20:01Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…odule

codecov-commenter · 2024-04-16T13:01:49Z

Codecov Report

Attention: Patch coverage is 27.91328% with 266 lines in your changes are missing coverage. Please review.

Project coverage is 30.80%. Comparing base (a5508ac) to head (f323822).
Report is 585 commits behind head on main.

❗ Current head f323822 differs from pull request most recent head 2e2ec00. Consider uploading reports for the commit 2e2ec00 to get more accurate results

Files	Patch %	Lines
...cloud/agent/manager/ClusteredAgentManagerImpl.java	7.89%	105 Missing ⚠️
...java/com/cloud/agent/manager/AgentManagerImpl.java	21.34%	70 Missing ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java	30.35%	39 Missing ⚠️
...ain/java/com/cloud/agent/manager/AgentAttache.java	31.81%	14 Missing and 1 partial ⚠️
...va/com/cloud/agent/manager/DirectAgentAttache.java	47.61%	11 Missing ⚠️
...com/cloud/agent/manager/ClusteredAgentAttache.java	0.00%	9 Missing ⚠️
...tack/engine/orchestration/NetworkOrchestrator.java	83.33%	5 Missing ⚠️
...ack/engine/orchestration/DataMigrationUtility.java	0.00%	3 Missing ⚠️
...tack/engine/orchestration/StorageOrchestrator.java	0.00%	3 Missing ⚠️
...a/com/cloud/agent/manager/SynchronousListener.java	33.33%	1 Missing and 1 partial ⚠️
... and 3 more

Additional details and impacted files

@@              Coverage Diff              @@
##               main    #8742       +/-   ##
=============================================
+ Coverage     13.17%   30.80%   +17.63%     
- Complexity     9214    33528    +24314     
=============================================
  Files          2725     5397     +2672     
  Lines        258235   379186   +120951     
  Branches      40249    55175    +14926     
=============================================
+ Hits          34013   116825    +82812     
- Misses       219913   246935    +27022     
- Partials       4309    15426    +11117

Flag	Coverage Δ
simulator-marvin-tests	`24.27% <27.37%> (?)`
uitests	`4.34% <ø> (?)`
unit-tests	`16.89% <1.08%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

JoaoJandre

CLGTM, hope I didn't miss anything 😅

DaanHoogland · 2024-04-22T07:34:23Z

@blueorangutan package

blueorangutan · 2024-04-22T07:36:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

DaanHoogland · 2024-04-22T07:39:15Z

CLGTM, hope I didn't miss anything 😅

that you'll have to test ;)

the build fails @KlausDornsbach in engine according to https://github.com/apache/cloudstack/actions/runs/8757275560/job/24037204263?pr=8742#step:7:17606 , afraid you'll have to take another look :(

blueorangutan · 2024-04-22T08:27:07Z

Packaging result [SF]: ✖️ el7 ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 9356

DaanHoogland · 2024-04-23T06:48:29Z

@blueorangutan package

blueorangutan · 2024-04-23T06:50:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-04-23T08:09:59Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9370

DaanHoogland · 2024-04-23T10:10:31Z

@blueorangutan test

blueorangutan · 2024-04-23T10:12:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-04-24T01:03:02Z

[SF] Trillian test result (tid-9971)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 51722 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8742-t9971-kvm-centos7.zip
Smoke tests completed. 127 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	302.27	test_events_resource.py
test_01_events_resource	`Error`	302.29	test_events_resource.py
test_04_deploy_vm_for_other_user_and_test_vm_operations	`Failure`	90.14	test_network_permissions.py
ContextSuite context=TestNetworkPermissions>:teardown	`Error`	1.38	test_network_permissions.py

DaanHoogland

clgtm and smoke tests pass

Co-authored-by: klaus.freitas.scclouds <klaus.freitas@scclouds.com.br>

klaus.freitas.scclouds added 3 commits February 16, 2024 16:08

Refactor cloud-engine-orchestration module logs for Log4j2

db778e0

adding some more log refactor

f0231b8

some more log refactoring for cloud-engine-orchestration module

2cedec5

boring-cyborg Bot added component:agent component:compute component:orchestration labels Mar 4, 2024

Merge branch 'main' into log4j2-refactor-cloud-engine-orchestration-m…

2c10459

…odule

KlausDornsbach marked this pull request as ready for review March 4, 2024 15:27

github-actions Bot added the status:has-conflicts label Mar 8, 2024

Merge branch 'main' into log4j2-refactor-cloud-engine-orchestration-m…

f323822

…odule

github-actions Bot added status:has-conflicts and removed status:has-conflicts labels Mar 8, 2024

JoaoJandre self-requested a review March 20, 2024 12:50

Merge branch 'main' into log4j2-refactor-cloud-engine-orchestration-m…

b677fd4

…odule

github-actions Bot removed the status:has-conflicts label Apr 16, 2024

JoaoJandre reviewed Apr 16, 2024

View reviewed changes

addressing comments

af2bfb7

JoaoJandre reviewed Apr 19, 2024

View reviewed changes

Comment thread ...hestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java Outdated

Comment thread ...hestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java Outdated

addressing other comments

3f0e309

JoaoJandre approved these changes Apr 19, 2024

View reviewed changes

fixing typo

2e2ec00

DaanHoogland added the status:needs-testing label Apr 23, 2024

DaanHoogland approved these changes Apr 24, 2024

View reviewed changes

DaanHoogland merged commit e330d76 into apache:main Apr 24, 2024

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request May 3, 2024

Log4j2 refactor cloud engine orchestration module (apache#8742)

b64bde0

Co-authored-by: klaus.freitas.scclouds <klaus.freitas@scclouds.com.br>

DaanHoogland removed the status:needs-testing label May 7, 2024

DaanHoogland added this to the 4.20.0.0 milestone May 7, 2024

Conversation

KlausDornsbach commented Mar 4, 2024

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

How Has This Been Tested?

Uh oh!

codecov Bot commented Mar 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Mar 8, 2024

Uh oh!

github-actions Bot commented Mar 13, 2024

Uh oh!

codecov-commenter commented Apr 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JoaoJandre left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Apr 22, 2024

Uh oh!

blueorangutan commented Apr 22, 2024

Uh oh!

DaanHoogland commented Apr 22, 2024

Uh oh!

blueorangutan commented Apr 22, 2024

Uh oh!

DaanHoogland commented Apr 23, 2024

Uh oh!

blueorangutan commented Apr 23, 2024

Uh oh!

blueorangutan commented Apr 23, 2024

Uh oh!

DaanHoogland commented Apr 23, 2024

Uh oh!

blueorangutan commented Apr 23, 2024

Uh oh!

blueorangutan commented Apr 24, 2024

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Mar 5, 2024 •

edited

Loading

codecov-commenter commented Apr 16, 2024 •

edited

Loading