Internal Server Error on what-if analysis after upgrading to 2.28.0 from 2.26.0

[Gordonby]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
Performing a what-if analysis results in an internal server error, i'm unable to decipher the problem. The actual deployment then continues to works fine on 2.28.0.

Reverting to AZ 2.26.0 results in the what-if and the deployment both working.

See the full What-If debug log here: https://github.com/Azure/Aks-Construction/runs/3863296553?check_suite_focus=true#step:11:146

DEBUG: cli.azure.cli.core.sdk.policies: ***"status":"Failed","error":***"code":"InternalServerError","message":"Encountered internal server error while processing the deployment what-if request. Diagnostic information: timestamp '20211011T200722Z', scope '***', tracking id '74f6c7cc-d330-4264-ad96-77c3a8ae7f55', request correlation id '2fd86693-ec6c-4931-9c2a-7e8b413496d5'."***
DEBUG: cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
DEBUG: cli.azure.cli.core.util: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 772, in what_if_deploy_arm_template_at_resource_group
    return _what_if_deploy_arm_template_at_resource_group_core(cmd, resource_group_name,
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 795, in _what_if_deploy_arm_template_at_resource_group_core
    what_if_result = _what_if_deploy_arm_template_core(cmd.cli_ctx, what_if_poller, no_pretty_print, exclude_change_types)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 897, in _what_if_deploy_arm_template_core
    raise CLIError(err_message)
knack.util.CLIError: InternalServerError - Encountered internal server error while processing the deployment what-if request. Diagnostic information: timestamp '20211011T200722Z', scope '***', tracking id '74f6c7cc-d330-4264-ad96-77c3a8ae7f55', request correlation id '2fd86693-ec6c-4931-9c2a-7e8b413496d5'.


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/arm.py", line 109, in handle_template_based_exception
    raise CLIError(ex.inner_exception.error.message)
AttributeError: 'CLIError' object has no attribute 'inner_exception'`

To Reproduce
Use the bicep files and the parameter file.
Run the following command, substituting two subnet resource id's and an arbitary resource name.

az deployment group what-if --debug -f bicep/main.bicep -g $RG -p .github/workflows_dep/AksDeploy-ByoVnet.parameters.json -p resourceName=$RESNAME byoAKSSubnetId=*** byoAGWSubnetId=*** 

Expected behavior
I'd like a clear error message of what's actually failed, or for it just to work as it did in the previous CLI version.

Environment summary
GitHub Az CLI Action.

Additional context
https://github.com/Azure/Aks-Construction/runs/3863296553?check_suite_focus=true#step:11:146

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

Issue Details

---

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
Performing a what-if analysis results in an internal server error, i'm unable to decipher the problem. The actual deployment then continues to works fine on 2.28.0.

Reverting to AZ 2.26.0 results in the what-if and the deployment both working.

See the full What-If debug log here: https://github.com/Azure/Aks-Construction/runs/3863296553?check_suite_focus=true#step:11:146

DEBUG: cli.azure.cli.core.sdk.policies: ***"status":"Failed","error":***"code":"InternalServerError","message":"Encountered internal server error while processing the deployment what-if request. Diagnostic information: timestamp '20211011T200722Z', scope '***', tracking id '74f6c7cc-d330-4264-ad96-77c3a8ae7f55', request correlation id '2fd86693-ec6c-4931-9c2a-7e8b413496d5'."***
DEBUG: cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
DEBUG: cli.azure.cli.core.util: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 772, in what_if_deploy_arm_template_at_resource_group
    return _what_if_deploy_arm_template_at_resource_group_core(cmd, resource_group_name,
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 795, in _what_if_deploy_arm_template_at_resource_group_core
    what_if_result = _what_if_deploy_arm_template_core(cmd.cli_ctx, what_if_poller, no_pretty_print, exclude_change_types)
  File "/usr/local/lib/python3.9/site-packages/azure/cli/command_modules/resource/custom.py", line 897, in _what_if_deploy_arm_template_core
    raise CLIError(err_message)
knack.util.CLIError: InternalServerError - Encountered internal server error while processing the deployment what-if request. Diagnostic information: timestamp '20211011T200722Z', scope '***', tracking id '74f6c7cc-d330-4264-ad96-77c3a8ae7f55', request correlation id '2fd86693-ec6c-4931-9c2a-7e8b413496d5'.


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/azure/cli/core/commands/arm.py", line 109, in handle_template_based_exception
    raise CLIError(ex.inner_exception.error.message)
AttributeError: 'CLIError' object has no attribute 'inner_exception'`

To Reproduce
Use the bicep files and the parameter file.
Run the following command, substituting two subnet resource id's and an arbitary resource name.

az deployment group what-if --debug -f bicep/main.bicep -g $RG -p .github/workflows_dep/AksDeploy-ByoVnet.parameters.json -p resourceName=$RESNAME byoAKSSubnetId=*** byoAGWSubnetId=*** 

Expected behavior
I'd like a clear error message of what's actually failed, or for it just to work as it did in the previous CLI version.

Environment summary
GitHub Az CLI Action.

Additional context
https://github.com/Azure/Aks-Construction/runs/3863296553?check_suite_focus=true#step:11:146

Author:	Gordonby
Assignees:	-
Labels:	Service Attention, ARM, needs-triage
Milestone:	-

[yonzhan]

route to service team

[zhoxing-ms]

@shenglol Could you please help to have a look at this issue?

[Gordonby]

I can confirm this is also an issue on 2.29.0

[Gordonby]

@shenglol - any thoughts?

https://github.com/Azure/Aks-Construction/runs/3909480699?check_suite_focus=true#step:10:29

[shenglol]

I'm still trying to figure out where the internal error comes from. It doesn't seem to be returned by the What-If service.

[shenglol]

Found the root cause. This is indeed a bug in the What-If engine. We added normalization process for Azure KeyVault access policies in 2021-01-01, but there's a case we failed to handle. I'm going to fix it.