CredentialUnavailableError error

[LiliDeng]

Describe the bug

We use below task in pipeline file.

- task: AzureCLI@2
        displayName: "Run One Sriov test"
        inputs:
          azureSubscription: 'name'
          scriptType: ps
          powerShellErrorActionPreference: 'silentlyContinue'
          scriptLocation: inlineScript
          inlineScript: |

And in the inlineScript we launch command to run python code, we always can see below error, I am sure the service connection has enough permission to operate the resources in the azure subscription.

2024-05-08 01:57:52.221[4124][ERROR] lisa.runner[0] case failed
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 141, in _run_command
    return subprocess.check_output(args, **kwargs)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmd', '/c', 'az account get-access-token --output json --resource https://management.azure.com/']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\lisa\lisa\runners\lisa_runner.py", line 295, in _deploy_environment_task
    self.platform.deploy_environment(environment)
  File "D:\a\_work\1\s\lisa\lisa\platform_.py", line 184, in deploy_environment
    self._deploy_environment(environment, log)
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 638, in _deploy_environment
    raise identifier
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 609, in _deploy_environment
    location, deployment_parameters = self._create_deployment_parameters(
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 1233, in _create_deployment_parameters
    node_arm_parameters = self._create_node_arm_parameters(node.capability, log)
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 1488, in _create_node_arm_parameters
    self._get_sig_os_disk_size(arm_parameters.shared_gallery),
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 2540, in _get_sig_os_disk_size
    found_image = self._get_sig_version(shared_image)
  File "D:\a\_work\1\s\lisa\lisa\sut_orchestrator\azure\platform_.py", line 2487, in _get_sig_version
    sig_version = compute_client.gallery_image_versions.get(
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\core\tracing\decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\mgmt\compute\v2022_03_03\operations\_operations.py", line 3580, in get
    pipeline_response: PipelineResponse = self._client._pipeline.run(  # pylint: disable=protected-access

Related command

'az account get-access-token --output json --resource'

Errors

see above

Issue script & Debug output

see above

Expected behavior

see above

Environment Summary

WARNING: You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade'
azure-cli 2.59.0 *

core 2.59.0 *
telemetry 1.1.0

Extensions:
azure-devops 1.0.0

Dependencies:
msal 1.27.0
azure-mgmt-resource 23.1.0b2

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[LiliDeng]

@jiasli
After changing azure-identity-1.9.0 into azure-identity-1.16.0, I saw below error. But I used the MSI with federated tokens have higher time limit. https://aka.ms/azdo-rm-workload-identity-manual, it is quite urgent for us, since we need get rid of the client secret of service principle.

  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\core\pipeline\policies\_redirect.py", line 197, in send
    response = self.next.send(request)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\core\pipeline\policies\_retry.py", line 531, in send
    response = self.next.send(request)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\core\pipeline\policies\_authentication.py", line 124, in send
    self.on_request(request)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\core\pipeline\policies\_authentication.py", line 99, in on_request
    self._token = self._credential.get_token(*self._scopes)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\identity\_credentials\default.py", line 219, in get_token
    token = self._successful_credential.get_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\identity\_internal\decorators.py", line 33, in wrapper
    token = fn(*args, **kwargs)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 112, in get_token
    output = _run_command(command, self._process_timeout)
  File "C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\site-packages\azure\identity\_credentials\azure_cli.py", line 215, in _run_command
    raise ClientAuthenticationError(message=message) from ex
azure.core.exceptions.ClientAuthenticationError: ERROR: AADSTS700024: Client assertion is not within its valid time range. Current time: 2024-05-09T02:10:40.3190445Z, assertion valid from 2024-05-09T01:55:09.0000000Z, expiry time of assertion 2024-05-09T02:05:09.0000000Z. Review the documentation at https://docs.microsoft.com/azure/active-directory/develop/active-directory-certificate-credentials . Trace ID: 7ea7efe9-2c80-47d7-b82b-b51b5902be00 Correlation ID: 25cf5e33-3808-473b-9266-3748d846bedd Timestamp: 2024-05-09 02:10:40Z
Interactive authentication is needed. Please run:
az login

[LiliDeng]

after upgrade az cli into 2.60.0 by running az upgrade --yes in task: AzureCLI@2, see a new issue

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: Authority.__init__() got an unexpected keyword argument 'oidc_authority_url'
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 664, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 701, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 334, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/profile/custom.py", line 97, in account_clear
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/_profile.py", line 296, in logout_all
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/auth/identity.py", line 210, in logout_all_users
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/auth/identity.py", line 119, in _msal_app
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 1827, in __init__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\msal/application.py", line 516, in __init__
TypeError: Authority.__init__() got an unexpected keyword argument 'oidc_authority_url'

[jiasli]

The call stack provided in #28915 (comment) is trimmed. Please share the full call stack.

But I used the MSI with federated tokens have higher time limit.

It doesn't matter if the access token of a managed identity lasts longer. The OIDC token itself expires after 10 minuts:

Current time: 2024-05-09T02:10:40.3190445Z, assertion valid from 2024-05-09T01:55:09.0000000Z, expiry time of assertion 2024-05-09T02:05:09.0000000Z.

This issue is tracked by #28737.

TypeError: Authority.__init__() got an unexpected keyword argument 'oidc_authority_url'

I guess you are hitting #20990.

[realsdx]

We are facing this issue on AzureCLI@2 task, (which currently uses Azure CLI 2.60.0 and masl 1.28.0)
Also, wanted to know if there is any quick workaround we can do to fix it temporarily @jiasli ?

Traceback On pipeline

2024-05-07T14:48:56.0522913Z Traceback (most recent call last):
2024-05-07T14:48:56.0523329Z   File "/home/vsts/work/1/s/lisa/lisa/testsuite.py", line 798, in __run_case
2024-05-07T14:48:56.0523678Z     _call_with_retry_and_timeout(
2024-05-07T14:48:56.0524029Z   File "/home/vsts/work/1/s/lisa/lisa/testsuite.py", line 64, in _call_with_retry_and_timeout
2024-05-07T14:48:56.0524366Z     retry_call(
2024-05-07T14:48:56.0525015Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/retry/api.py", line 101, in retry_call
2024-05-07T14:48:56.0525468Z     return __retry_internal(partial(f, *args, **kwargs), exceptions, tries, delay, max_delay, backoff, jitter, logger)
2024-05-07T14:48:56.0526048Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/retry/api.py", line 33, in __retry_internal
2024-05-07T14:48:56.0526377Z     return f()
2024-05-07T14:48:56.0526877Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/func_timeout/dafunc.py", line 108, in func_timeout
2024-05-07T14:48:56.0527293Z     raise_exception(exception)
2024-05-07T14:48:56.0527806Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/func_timeout/py3_raise.py", line 7, in raise_exception
2024-05-07T14:48:56.0528154Z     raise exception[0] from None
2024-05-07T14:48:56.0528473Z   File "/home/vsts/work/1/s/lisa/lisa/testsuite.py", line 519, in wrapper
2024-05-07T14:48:56.0529172Z     func(*args, **parameters)
2024-05-07T14:48:56.0529510Z   File "/home/vsts/work/1/s/lisa/microsoft/testsuites/core/serial_console.py", line 41, in verify_serial_console
2024-05-07T14:48:56.0529899Z     _ = serial_console.read()
2024-05-07T14:48:56.0530416Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/decorator.py", line 232, in fun
2024-05-07T14:48:56.0530842Z     return caller(func, *(extras + args), **kw)
2024-05-07T14:48:56.0531663Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/retry/api.py", line 73, in retry_decorator
2024-05-07T14:48:56.0532570Z     return __retry_internal(partial(f, *args, **kwargs), exceptions, tries, delay, max_delay, backoff, jitter,
2024-05-07T14:48:56.0533202Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/retry/api.py", line 33, in __retry_internal
2024-05-07T14:48:56.0533586Z     return f()
2024-05-07T14:48:56.0533917Z   File "/home/vsts/work/1/s/lisa/lisa/sut_orchestrator/azure/features.py", line 338, in read
2024-05-07T14:48:56.0534246Z     output = self._read()
2024-05-07T14:48:56.0534678Z   File "/home/vsts/work/1/s/lisa/lisa/sut_orchestrator/azure/features.py", line 376, in _read
2024-05-07T14:48:56.0535093Z     self._initialize_serial_console(port_id=self.DEFAULT_SERIAL_PORT_ID)
2024-05-07T14:48:56.0535496Z   File "/home/vsts/work/1/s/lisa/lisa/sut_orchestrator/azure/features.py", line 454, in _initialize_serial_console
2024-05-07T14:48:56.0536606Z     serial_ports = self._serial_port_operations.list(
2024-05-07T14:48:56.0537432Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/mgmt/serialconsole/operations/_serial_ports_operations.py", line 101, in list
2024-05-07T14:48:56.0537947Z     pipeline_response = self._client._pipeline.run(request, stream=False, **kwargs)
2024-05-07T14:48:56.0538488Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 230, in run
2024-05-07T14:48:56.0538894Z     return first_node.send(pipeline_request)
2024-05-07T14:48:56.0539413Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
2024-05-07T14:48:56.0540157Z     response = self.next.send(request)
2024-05-07T14:48:56.0540667Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
2024-05-07T14:48:56.0541039Z     response = self.next.send(request)
2024-05-07T14:48:56.0541541Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
2024-05-07T14:48:56.0541915Z     response = self.next.send(request)
2024-05-07T14:48:56.0542200Z   [Previous line repeated 2 more times]
2024-05-07T14:48:56.0544190Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/mgmt/core/policies/_base.py", line 46, in send
2024-05-07T14:48:56.0544584Z     response = self.next.send(request)
2024-05-07T14:48:56.0545153Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/policies/_redirect.py", line 197, in send
2024-05-07T14:48:56.0545574Z     response = self.next.send(request)
2024-05-07T14:48:56.0546081Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/policies/_retry.py", line 531, in send
2024-05-07T14:48:56.0546470Z     response = self.next.send(request)
2024-05-07T14:48:56.0547009Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/policies/_authentication.py", line 124, in send
2024-05-07T14:48:56.0547376Z     self.on_request(request)
2024-05-07T14:48:56.0547926Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/core/pipeline/policies/_authentication.py", line 99, in on_request
2024-05-07T14:48:56.0548339Z     self._token = self._credential.get_token(*self._scopes)
2024-05-07T14:48:56.0549085Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/identity/_credentials/default.py", line 169, in get_token
2024-05-07T14:48:56.0549760Z     token = self._successful_credential.get_token(*scopes, **kwargs)
2024-05-07T14:48:56.0565106Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/identity/_internal/decorators.py", line 32, in wrapper
2024-05-07T14:48:56.0567902Z     token = fn(*args, **kwargs)
2024-05-07T14:48:56.0571239Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/identity/_credentials/azure_cli.py", line 73, in get_token
2024-05-07T14:48:56.0571753Z     output = _run_command(command)
2024-05-07T14:48:56.0572390Z   File "/home/vsts/work/1/s/lisa/.venv/lib/python3.10/site-packages/azure/identity/_credentials/azure_cli.py", line 147, in _run_command
2024-05-07T14:48:56.0572891Z     raise CredentialUnavailableError(message=NOT_LOGGED_IN)
2024-05-07T14:48:56.0573471Z azure.identity._exceptions.CredentialUnavailableError: Please run 'az login' to set up an account