NO PROXY: az aks install-cli: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate

[bitsofinfo]

Running from windows powershell az aks install-cli

windows version below.

az login works just fine

az aks install-cli
The detected architecture is 'amd64', which will be regarded as 'amd64' and the corresponding binary will be downloaded. If there is any problem, please download the appropriate binary by yourself.
The command failed with an unexpected error. Here is the traceback:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
Traceback (most recent call last):
  File "urllib\request.py", line 1348, in do_open
  File "http\client.py", line 1282, in request
  File "http\client.py", line 1328, in _send_request
  File "http\client.py", line 1277, in endheaders
  File "http\client.py", line 1037, in _send_output
  File "http\client.py", line 975, in send
  File "http\client.py", line 1454, in connect
  File "ssl.py", line 513, in wrap_socket
  File "ssl.py", line 1071, in _create
  File "ssl.py", line 1342, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/acs/custom.py", line 1432, in k8s_install_cli
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/acs/custom.py", line 1547, in k8s_install_kubectl
  File "urllib\request.py", line 216, in urlopen
  File "urllib\request.py", line 519, in open
  File "urllib\request.py", line 536, in _open
  File "urllib\request.py", line 496, in _call_chain
  File "urllib\request.py", line 1391, in https_open
  File "urllib\request.py", line 1351, in do_open
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
To open a new issue, please run `az feedback`

WindowsBuildLabEx                                       : 22621.1.amd64fre.ni_release.220506-1250
WindowsCurrentVersion                                   : 6.3
WindowsEditionId                                        : Professional
WindowsInstallationType                                 : Client
WindowsInstallDateFromRegistry                          : 1/26/2023 10:55:07 PM
WindowsProductId                                        : 00331-10000-00001-AA589
WindowsProductName                                      : Windows 10 Pro
WindowsRegisteredOrganization                           :
WindowsRegisteredOwner                                  :
WindowsSystemRoot                                       : C:\Windows
WindowsVersion                                          : 2009

[bitsofinfo]

This works fine btw: python -c "import requests; print(requests.get('https://login.microsoftonline.com/').status_code)"

[yonzhan]

route to CXP team

[weinong]

did you take a look at #23282?

[levi106]

I could reproduce this issue with the following python script in a Windows Server 2022 environment.

>>> from urllib.request import urlopen
>>> import socket, ssl
>>> ctx = ssl.create_default_context()
>>> urlopen("https://storage.googleapis.com/kubernetes-release/release/stable.txt", context=ctx)
Traceback (most recent call last):
  File "urllib\request.py", line 1348, in do_open
  File "http\client.py", line 1282, in request
  File "http\client.py", line 1328, in _send_request
  File "http\client.py", line 1277, in endheaders
  File "http\client.py", line 1037, in _send_output
  File "http\client.py", line 975, in send
  File "http\client.py", line 1454, in connect
  File "ssl.py", line 513, in wrap_socket
  File "ssl.py", line 1071, in _create
  File "ssl.py", line 1342, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "urllib\request.py", line 216, in urlopen
  File "urllib\request.py", line 519, in open
  File "urllib\request.py", line 536, in _open
  File "urllib\request.py", line 496, in _call_chain
  File "urllib\request.py", line 1391, in https_open
  File "urllib\request.py", line 1351, in do_open
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
>>> len(ctx.get_ca_certs())
13
>>> [print(x['subject'][-1][0]) for x in ctx.get_ca_certs()]
('commonName', 'DST Root CA X3')
('commonName', 'ISRG Root X1')
('commonName', 'Baltimore CyberTrust Root')
('organizationalUnitName', 'Class 3 Public Primary Certification Authority')
('commonName', 'Microsoft Root Certificate Authority')
('commonName', 'DigiCert Global Root CA')
('commonName', 'DigiCert Global Root G2')
('commonName', 'Microsoft Root Certificate Authority 2010')
('commonName', 'Microsoft Root Certificate Authority 2011')
('commonName', 'Microsoft ECC TS Root Certificate Authority 2018')
('commonName', 'Microsoft Time Stamp Root Certificate Authority 2014')
('commonName', 'Microsoft ECC Product Root Certificate Authority 2018')
('organizationalUnitName', 'www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign')
[None, None, None, None, None, None, None, None, None, None, None, None, None]

The reason for the exception appears to be the lack of GlobalSign Root CA.

After browsing the above url with edge browser, no exception occurs.

>>> ctx = ssl.create_default_context()
>>> len(ctx.get_ca_certs())
15
>>> [print(x['subject'][-1][0]) for x in ctx.get_ca_certs()]
('commonName', 'DST Root CA X3')
('commonName', 'ISRG Root X1')
('commonName', 'GlobalSign Root CA')
('commonName', 'Baltimore CyberTrust Root')
('organizationalUnitName', 'Class 3 Public Primary Certification Authority')
('commonName', 'Microsoft Root Certificate Authority')
('commonName', 'DigiCert Global Root CA')
('commonName', 'DigiCert Global Root G2')
('organizationalUnitName', 'Starfield Class 2 Certification Authority')
('commonName', 'Microsoft Root Certificate Authority 2010')
('commonName', 'Microsoft Root Certificate Authority 2011')
('commonName', 'Microsoft ECC TS Root Certificate Authority 2018')
('commonName', 'Microsoft ECC Product Root Certificate Authority 2018')
('organizationalUnitName', 'www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign')
('commonName', 'Microsoft Time Stamp Root Certificate Authority 2014')
[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]
>>> urlopen("https://storage.googleapis.com/kubernetes-release/release/stable.txt", context=ctx)
<http.client.HTTPResponse object at 0x011847C0>

[artisticcheese]

I could reproduce this issue with the following python script in a Windows Server 2022 environment.

>>> from urllib.request import urlopen
>>> import socket, ssl
>>> ctx = ssl.create_default_context()
>>> urlopen("https://storage.googleapis.com/kubernetes-release/release/stable.txt", context=ctx)
Traceback (most recent call last):
  File "urllib\request.py", line 1348, in do_open
  File "http\client.py", line 1282, in request

This worked for me ONLY I after I visited https://storage.googleapis.com/kubernetes-release/release/stable.txt in Internet Explorer in Windows 2022. Visiting it in Edge did not do the trick. In fact certificate chain looked different when visiting it in Edge vs Internet Explorer