Deployments failing on App GW NSG rule validation after initial deployment

[biggles007]

This is autogenerated. Please review and update as needed.

Describe the bug

After initial greenfield deployment, future deployments fail on Application Gateway NSG validation, previously reported in #21256 and was seemingly working again in a previous version.

Command Name
az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup\\\",\\r\\n    \\\"message\\\": \\\"Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.\\\",\\r\\n    \\\"details\\\": []\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behaviour. Note that argument values have been redacted, as they may contain sensitive information.

Run the deployment a second time

Expected Behaviour

Deployment should succeed

Environment Summary

Windows-10-10.0.19043-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.36.0

Additional Context

[yonzhan]

ARM

[zhoxing-ms]

Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.

@bigglesuk69 Could you see if this error message is helpful to you?

[biggles007]

@zhoxing-ms the rules are on the NSG, I have validated they are correct, so the error message is wrong. The issue is somewhere in the validation. To test I did the following.

Deleted the App GW, ran the deployment, successful
Run the deployment a second time, error is shown.
At all times the NSG has the correct rule.

NSG is deployed with no rules, then we inject in the pre-requisite rules using the Microsoft.Network/networkSecurityGroups/securityRules resource. If the validation only checking the Microsoft.Network/networkSecurityGroups resource for the rules? If so, that is a bug.

[biggles007]

Some further information on this issue. I have a master bicep template that calls child modules for the deployments. So the vnet/NSG is deployed in one deployment, and the app GW in another. So what is happening is:

• vnet deployment runs first time and succeeds (no AppGW deployed)
• AppGW deployment runs successfully once the vnet deployment has completed
• vnet deployment fails on all further runs

[zhoxing-ms]

This error message is returned by the REST service, so this is a service issue

Thank you for your feedback. This has been routed to the support team for assistance.

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @josephkwchan, @jennyhunter-msft.

Issue Details

---

This is autogenerated. Please review and update as needed.

Describe the bug

After initial greenfield deployment, future deployments fail on Application Gateway NSG validation, previously reported in #21256 and was seemingly working again in a previous version.

Command Name
az deployment group create

Errors:

{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"BadRequest\",\r\n            \"message\": \"{\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup\\\",\\r\\n    \\\"message\\\": \\\"Network security group /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/networkSecurityGroups/appgw blocks incoming internet traffic on ports 65200 - 65535 to subnet /subscriptions/xxxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/virtualNetworks/vnet-weu-aksaccel/subnets/appgw, associated with Application Gateway /subscriptions/xxx/resourceGroups/rg-weu-aksaccelerator-1/providers/Microsoft.Network/applicationGateways/appgw-weu-aksaccel. This is not permitted for Application Gateways that have V2 Sku.\\\",\\r\\n    \\\"details\\\": []\\r\\n  }\\r\\n}\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}

To Reproduce:

Steps to reproduce the behaviour. Note that argument values have been redacted, as they may contain sensitive information.

Run the deployment a second time

Expected Behaviour

Deployment should succeed

Environment Summary

Windows-10-10.0.19043-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.36.0

Additional Context

Author:	bigglesuk69
Assignees:	zhoxing-ms
Labels:	Service Attention, ARM, customer-reported, Auto-Assign
Milestone:	Backlog