{AKS} Fix DCR create/update for container network logs and high log scale mode#9667
{AKS} Fix DCR create/update for container network logs and high log scale mode#9667carlotaarvela wants to merge 26 commits intoAzure:mainfrom
Conversation
️✔️Azure CLI Extensions Breaking Change Test
|
|
Hi @carlotaarvela, |
|
Thank you for your contribution! We will review the pull request and get back to you soon. |
|
The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR. Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions). pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>
|
CodeGen Tools Feedback CollectionThank you for using our CodeGen tool. We value your feedback, and we would like to know how we can improve our product. Please take a few minutes to fill our codegen survey |
|
955792f to
43c3a1f
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes AKS az aks create / az aks update behavior so the Azure Monitor Data Collection Rule (DCR) stays in sync when container network logs (CNL) and/or high log scale mode (HLSM) flags are used, and adds unit tests to cover those scenarios.
Changes:
- Update create/update decorators to correctly trigger DCR/DCRA creation/update when CNL/HLSM-related flags are provided.
- Add/extend validation and casing handling for monitoring addon profiles (
omsagentvsomsAgent) in relevant paths. - Add comprehensive unit tests for create/update flows covering CNL/HLSM combinations and edge cases.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/aks-preview/setup.py |
Bumps extension version to 19.0.0b26. |
src/aks-preview/azext_aks_preview/managed_cluster_decorator.py |
Adjusts create/update postprocessing and validation so DCR/DCRA are updated for CNL/HLSM scenarios, including addon key casing handling. |
src/aks-preview/azext_aks_preview/tests/latest/test_managed_cluster_decorator.py |
Adds/extends unit tests for the new create/update behaviors and edge cases. |
src/aks-preview/HISTORY.rst |
Documents the fix and validation changes in release notes. |
You can also share your feedback on Copilot code review. Take the survey.
FumingZhang
left a comment
There was a problem hiding this comment.
Could you add some integration tests and run the existing tests to confirm that the change works as expected?
There was a problem hiding this comment.
Queued live test to validate the change.
- test_aks_create_with_azuremonitorlogs_and_cnl
- test_aks_update_enable_azuremonitorlogs_with_hlsm
- test_aks_create_with_retina_flow_logs_alias
- test_aks_update_enable_cnl_via_azuremonitorlogs
- test_aks_update_disable_azuremonitorlogs
- test_aks_update_standalone_enable_high_log_scale_mode
Could you please schedule another live test for cases involving the monitoring addon to ensure the change does not lead to any regression? I am relying on the test results to approve the change. @carlotaarvela
2959c0a to
598d64f
Compare
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
Re-queued live test |
|
Please rebase from main to pick up the fix for linter failure about The CI |
a8ba56e to
1c4f648
Compare
|
Live tests are passing |
|
Hi @carlotaarvela, please resolve merge conflicts |
f797bfd to
2faa27c
Compare
| * `az aks create/update`: Fix DCR not being created or updated when `--enable-container-network-logs`, `--enable-retina-flow-logs`, or `--enable-high-log-scale-mode` flags are used, ensuring the Data Collection Rule streams (e.g. `Microsoft-ContainerLogV2-HighScale`) are kept in sync. | ||
| * `az aks update`: Add validation for `--enable-high-log-scale-mode` on the update path requiring the monitoring addon with MSI authentication to be enabled. |
There was a problem hiding this comment.
Please also add the history notes under the Pending section in your new release version.
Related command
az aks create,az aks updateGeneral Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally? (pip install wheel==0.30.0required)For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.Description
Fixes the
az aks createandaz aks updatecommands so that the Data Collection Rule (DCR) is properly created or updated when container network logs (CNL) or high log scale mode (HLSM) flags are passed. Previously, the DCR was not being created/updated in these scenarios, causing streams likeMicrosoft-ContainerLogV2-HighScaleto be out of sync.Also fixes
--disable-azure-monitor-logsso it actually disables monitoring by settingazureMonitorProfile.containerInsights.enabled = False(the RP's source of truth), and handlesomsAgentvsomsagentkey casing inconsistencies from the Azure API.Changes
Shared utilities:
_get_monitoring_addon_key()helper to resolve the monitoring addon key from a cluster'saddon_profiles, handling bothomsagentandomsAgentkey variants returned by the API.Context (
AKSPreviewManagedClusterContext):get_enable_msi_auth_for_monitoring()to detect MSI auth on clusters whereservice_principal_profile.client_idis"msi"by checking the addon'suseAADAuthconfig instead of returningFalse.get_container_network_logs()andget_enable_high_log_scale_mode()to handle bothomsagentandomsAgentaddon key variants.MutuallyExclusiveArgumentError.enable_high_log_scale_modeto the special parameter defaults list so standalone--enable-high-log-scale-modepasses thecheck_raw_parametersgate.Create path (
AKSPreviewManagedClusterCreateDecorator):_setup_azure_monitor_logs()(pre-PUT), matching the base classbuild_monitoring_addon_profilepattern. DCRA is still created in postprocessing._should_create_dcra()and_is_cnl_or_hlsm_changing()helpers to detect when CNL/HLSM flags require DCR/DCRA creation._postprocess_monitoring_enable()and_postprocess_monitoring_disable()into reusable methods.Update path (
AKSPreviewManagedClusterUpdateDecorator):update_monitoring_profile_flow_logs(): Added base class delegation (super().update_monitoring_profile_flow_logs(mc)) for CLI >= 2.84.0 compatibility, and guarded the call site inupdate_mc_profile_preview()to avoid double execution.--enable-container-network-logsor--enable-retina-flow-logsis passed, themonitoring_addon_postprocessing_requiredintermediate is now set, triggering the DCR update viaensure_container_insights_for_monitoring.--enable-high-log-scale-mode: requires the monitoring addon with MSI auth to be enabled, with an escape hatch when--enable-azure-monitor-logsis passed simultaneously.--enable-high-log-scale-mode false: prevents disabling HLSM while CNL is still enabled._disable_azure_monitor_logs(): now setsmc.azure_monitor_profile.container_insights.enabled = False(the RP source of truth) and usesconfig = Noneinstead ofconfig = {}to match the base CLI behavior._get_monitoring_addon_key()for correct key resolution.Tests:
test_aks_create_with_azuremonitorlogs_and_cnl,test_aks_update_enable_azuremonitorlogs_with_hlsm,test_aks_create_with_retina_flow_logs_alias,test_aks_update_enable_cnl_via_azuremonitorlogs,test_aks_update_disable_azuremonitorlogs,test_aks_update_standalone_enable_high_log_scale_mode.Testing
pytest src/aks-preview/azext_aks_preview/tests/latest/test_managed_cluster_decorator.py -k "container_network_logs or high_log_scale_mode or enable_container_network or standalone_high_log or disable_azure_monitor or postprocessing_create_dcr or update_monitoring_profile_flow_logs"AZURE_TEST_RUN_LIVE=true pytest src/aks-preview/azext_aks_preview/tests/latest/test_aks_commands.py -k "test_aks_create_with_azuremonitorlogs_and_cnl or test_aks_update_enable_azuremonitorlogs_with_hlsm or test_aks_create_with_retina_flow_logs_alias or test_aks_update_enable_cnl_via_azuremonitorlogs or test_aks_update_disable_azuremonitorlogs or test_aks_update_standalone_enable_high_log_scale_mode"