az rest fails when an exception is returned as a response

[VISlavov]

Describe the bug

az rest fails when an exception is returned directly as a response. This didn't occur with older versions of the CLI. Unfortunately, I can't specify which version works, exactly, because I didn't take notice of the version I had before I upgrade.

Related command

az rest

Errors

The command failed with an unexpected error: "TypeError: Object of type HTTPError is not JSON serializable"
Rest of the traceback:

Suppress exception:
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 62, in <module>
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 55, in <module>
SystemExit: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/decorators.py", line 79, in _wrapped_func
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/telemetry.py", line 123, in generate_payload
  File "json\__init__.py", line 238, in dumps
  File "json\encoder.py", line 199, in encode
  File "json\encoder.py", line 257, in iterencode
  File "json\encoder.py", line 179, in default
TypeError: Object of type HTTPError is not JSON serializable

Issue script & Debug output

Script:
az rest --debug --verbose --method post --resource 00000000-0000-0000-0000-000000000000 --body '{ custom data here }' --url https://app.azurewebsites.net/
Debug log:

cli.knack.cli: Command arguments: ['rest', '--debug', '--verbose', '--method', 'post', '--resource', '00000000-0000-0000-0000-000000000000', '--body', '{ custom data here }', '--url', 'https://app.azurewebsites.net/']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
Enable VT mode.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x02B7B658>, <function OutputProducer.on_global_arguments at 0x02D7A898>, <function CLIQuery.on_global_arguments at 0x02D914F0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'rest': ['azure.cli.command_modules.util']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: util                      0.004         3         7
cli.azure.cli.core: Total (1)                 0.004         3         7
cli.azure.cli.core: Loaded 3 groups, 7 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : rest
cli.azure.cli.core: Command table: rest
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x04EDC7C0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\user\.azure\commands\2023-08-07.11-48-41.rest.13924.log'.
az_command_data_logger: command args: rest --debug --verbose --method {} --resource {} --body {} --url {}
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x04F06898>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x04F16808>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x04F16A00>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x02D7A8E0>, <function CLIQuery.handle_query_parameter at 0x02D91538>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x04F169B8>]
cli.azure.cli.core.util: Retrieving token for resource 00000000-0000-0000-0000-000000000000
cli.azure.cli.core.auth.persistence: build_persistence: location='C:\\Users\\user\\.azure\\msal_token_cache.bin', encrypt=True
cli.azure.cli.core.auth.binary_cache: load: C:\Users\user\.azure\msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/kerberos', 'tenant_region_scope': 'EU', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? False
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('00000000-0000-0000-0000-000000000000/.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: 00000000-0000-0000-0000-000000000000
cli.azure.cli.core.util: Request URL: 'https://app.azurewebsites.net/'
cli.azure.cli.core.util: Request method: 'POST'
cli.azure.cli.core.util: Request headers:
cli.azure.cli.core.util:     'User-Agent': 'python/3.10.10 (Windows-10-10.0.19044-SP0) AZURECLI/2.51.0 (MSI)'
cli.azure.cli.core.util:     'Accept-Encoding': 'gzip, deflate'
cli.azure.cli.core.util:     'Accept': '*/*'
cli.azure.cli.core.util:     'Connection': 'keep-alive'
cli.azure.cli.core.util:     'x-ms-client-request-id': '00000000-0000-0000-0000-000000000000'
cli.azure.cli.core.util:     'Content-Type': 'application/json'
cli.azure.cli.core.util:     'CommandName': 'rest'
cli.azure.cli.core.util:     'ParameterSetName': '--debug --verbose --method --resource --body --url'
cli.azure.cli.core.util:     'Authorization': 'Bearer ...'
cli.azure.cli.core.util:     'Content-Length': '353'
cli.azure.cli.core.util: Request body:
cli.azure.cli.core.util: { custom data here }
urllib3.connectionpool: Starting new HTTPS connection (1): app.azurewebsites.net:443
urllib3.connectionpool: https://app.azurewebsites.net:443 "POST / HTTP/1.1" 500 None
cli.azure.cli.core.util: Response status: 500
cli.azure.cli.core.util: Response headers:
cli.azure.cli.core.util:     'Date': 'Mon, 07 Aug 2023 10:48:41 GMT'
cli.azure.cli.core.util:     'Server': 'Kestrel'
cli.azure.cli.core.util:     'Transfer-Encoding': 'chunked'
cli.azure.cli.core.util:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.util: Response content:
cli.azure.cli.core.util: Exception: CustomException
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/util/custom.py", line 24, in rest_call
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/util.py", line 1010, in send_raw_request
azure.cli.core.azclierror.HTTPError: Internal Server Error(CustomException)
az_command_data_logger: Internal Server Error(CustomException)
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x04EDC8E0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 0.955 seconds (init: 0.327, invoke: 0.628)
cli.azure.cli.core.decorators: Suppress exception:
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 62, in <module>
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/__main__.py", line 55, in <module>
SystemExit: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/decorators.py", line 79, in _wrapped_func
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/telemetry.py", line 123, in generate_payload
  File "json\__init__.py", line 238, in dumps
  File "json\encoder.py", line 199, in encode
  File "json\encoder.py", line 257, in iterencode
  File "json\encoder.py", line 179, in default
TypeError: Object of type HTTPError is not JSON serializable

telemetry.main: Split cli events and extra events failure: the JSON object must be str, bytes or bytearray, not NoneType

Expected behavior

The CLI exits with code 0.

Environment Summary

azure-cli 2.51.0

core 2.51.0
telemetry 1.1.0

Dependencies:
msal 1.24.0b1
azure-mgmt-resource 23.1.0b2

Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb 7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thank you for your feedback. This has been routed to the support team for assistance.

[navba-MSFT]

@VISlavov Thanks for reaching out to us and reporting this issue. Could you please provide more details about your use case ? Which Azure Resource are you calling via Az REST cli command ? Is this a valid endpoint https://app.azurewebsites.net/ ? what is it expecting to return ? Please share the steps using which I can reproduce this issue at my end.

Awaiting your reply.