Install fails with certificate verify failed

[Brian2099]

I've been running Strawberry Perl in a windows environment since 2015. Twice a year, I upgrade the CPAN modules and if necessary, Strawberry Perl.

I'm developing on a Windows 10 laptop.
Strawberry Perl. As of 1/1/2024, ver 5.38.0.1. As of 6/27/2024 I have ver 5.38.2.2

Yesterday, June 27, 2024, I tried installing LWP::Protocol::https via the cpanm tool, but it failed several tests. The version it tried to load was 6.14.

I then did a force install. I tried running my perl script, but it abended with http error 500. Can't connect to aaa.bbb.com:443 (certificate verify failed)

Per google search results, I changed the parameter on my HTTP call to ssl_opts => { verify_hostname => 0 }, but that didn't work.

I also tried this line of code, but the results were the same: $ENV{'PERL_LWP_SSL_VERIFY_HOSTNAME'} = 0;

I verified that the cert for aaa.bbb.com is valid. I also ran a windows version of the Perl script and it ran fine.

I realized that my Strawberry Perl was a bit out of date, so installed the most recent version.

Then I uninstalled LWP::Protocol::https

Next, I tried installing LWP::Protocol::https again via the cpanm tool, but it failed the same tests as before.

Went to GitHub and downloaded LWP-Protocol-https-6.12.tar.gz, which is two releases back. Tried to do an install and got the same certificate error.

Any advice would be appreciated!

+++++++++++++++++++++++++++++++++++++++++++++++++
++
Here is the screen comments from the install:

C:\myStuff\Perl>cpanm LWP::Protocol::https
--> Working on LWP::Protocol::https
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/LWP-Protocol-https-6.14.tar.gz ... OK
Configuring LWP-Protocol-https-6.14 ... OK
Building and testing LWP-Protocol-https-6.14 ... FAIL
! Installing LWP::Protocol::https failed. See C:\Users\BRIAN@@@.cpanm\work\1719595378.23088\build.log for details. Retry with --force to force install it.

C:\myStuff\Perl>

+++++++++++++++++++++++++++++++++++++++++++++++++
++
Here is the build.log:

cpanm (App::cpanminus) 1.7047 on perl 5.038002 built for MSWin32-x64-multi-thread
Work directory is C:\Users\BRIAN@@@/.cpanm/work/1719595378.23088
You have make C:\Strawberry\c\bin\gmake.exe
You have LWP 6.72
Falling back to Archive::Tar 3.02
Searching LWP::Protocol::https () on cpanmetadb ...
--> Working on LWP::Protocol::https
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/LWP-Protocol-https-6.14.tar.gz
-> OK
Unpacking LWP-Protocol-https-6.14.tar.gz
Entering LWP-Protocol-https-6.14
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.58 ... Yes (7.70)
Configuring LWP-Protocol-https-6.14
Running Makefile.PL
Checking if your kit is complete...
Looks good
Generating a gmake-style Makefile
Writing Makefile for LWP::Protocol::https
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have File::Spec 0 ... Yes (3.88)
Checking if you have Test::Needs 0.002010 ... Yes (0.002010)
Checking if you have warnings 0 ... Yes (1.65)
Checking if you have Test::More 0.96 ... Yes (1.302198)
Checking if you have LWP::UserAgent 6.06 ... Yes (6.72)
Checking if you have ExtUtils::MakeMaker 0 ... Yes (7.70)
Checking if you have Test::RequiresInternet 0 ... Yes (0.05)
Checking if you have Socket 0 ... Yes (2.037)
Checking if you have LWP::Protocol::http 0 ... Yes (6.72)
Checking if you have IO::Socket::SSL 1.970 ... Yes (2.085)
Checking if you have base 0 ... Yes (2.27)
Checking if you have Net::HTTPS 6 ... Yes (6.23)
Checking if you have IO::Select 0 ... Yes (1.52)
Checking if you have File::Temp 0 ... Yes (0.2311)
Checking if you have IO::Socket::SSL::Utils 0 ... Yes (2.015)
Checking if you have strict 0 ... Yes (1.12)
Checking if you have IO::Socket::INET 0 ... Yes (1.52)
Building and testing LWP-Protocol-https-6.14
cp lib/LWP/Protocol/https.pm blib\lib\LWP\Protocol\https.pm
"C:\Strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/.t

Versions for all modules listed in MYMETA.json (including optional ones):

=== Configure Requires ===

Module Want Have

------------------- -------- --------

ExtUtils::MakeMaker any 7.70

perl 5.008001 5.038002

=== Configure Suggests ===

Module Want Have

-------- ------- ----

JSON::PP 2.27300 4.16

=== Build Requires ===

Module Want Have

------------------- ---- ----

ExtUtils::MakeMaker any 7.70

=== Test Requires ===

Module Want Have

---------------------- -------- --------

ExtUtils::MakeMaker any 7.70

File::Spec any 3.88

File::Temp any 0.2311

IO::Select any 1.52

IO::Socket::INET any 1.52

IO::Socket::SSL 1.970 2.085

IO::Socket::SSL::Utils any 2.015

LWP::UserAgent 6.06 6.72

Socket any 2.037

Test::More 0.96 1.302198

Test::Needs 0.002010 0.002010

Test::RequiresInternet any 0.05

perl 5.008001 5.038002

warnings any 1.65

=== Test Recommends ===

Module Want Have

---------- -------- --------

CPAN::Meta 2.120900 2.150010

=== Runtime Requires ===

Module Want Have

------------------- -------- --------

IO::Socket::SSL 1.970 2.085

LWP::Protocol::http any 6.72

LWP::UserAgent 6.06 6.72

Net::HTTPS 6 6.23

base any 2.27

perl 5.008001 5.038002

strict any 1.12

=== Other Modules ===

Module Have

----------- ----

Net::SSLeay 1.94

t/00-report-prereqs.t .. ok
# path to openssl: C:\Strawberry\c\bin\openssl.EXE
# stdout: OpenSSL 1.1.1q 5 Jul 2022
# Net::SSLeay::OPENSSL_VERSION_NUMBER() 0x1010111f
# IO::Select 1.52
# IO::Socket::INET 1.52
# IO::Socket::SSL 2.085
# IO::Socket::SSL::Utils 2.015
# Socket 2.037
t/diag.t ............... ok

#   Failed test 'success status'
#   at t/example.t line 19.

#   Failed test 'have header Client-SSL-Socket-Class'
#   at t/example.t line 29.

#   Failed test 'have header Client-SSL-Cipher'
#   at t/example.t line 44.

#   Failed test 'found expected document content'
#   at t/example.t line 46.
#                   'Can't connect to www.example.com:443 (certificate verify failed)
# 
# SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at C:/Strawberry/perl/vendor/lib/LWP/Protocol/http.pm line 50.
# '
#     doesn't match '(?^:Example Domain)'
# Looks like you failed 4 tests of 6.

Failed test 'Request GET https://www.example.com'

at t/example.t line 47.

#   Failed test 'success status'
#   at t/example.t line 54.
# Looks like you failed 1 test of 2.

Failed test 'Check for warnings from GET https://www.example.com (RT #81948)'

at t/example.t line 57.

Looks like you failed 2 tests of 2.

t/example.t ............
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/2 subtests
t/https_proxy.t ........ ok

Test Summary Report

t/example.t (Wstat: 512 (exited 2) Tests: 2 Failed: 2)
Failed tests: 1-2
Non-zero exit status: 2
Files=4, Tests=62, 3 wallclock secs ( 0.01 usr + 0.05 sys = 0.06 CPU)
Result: FAIL
Failed 1/4 test programs. 2/62 subtests failed.
gmake: *** [makefile:889: test_dynamic] Error 255
-> FAIL Installing LWP::Protocol::https failed. See C:\Users\BRIAN@@@.cpanm\work\1719595378.23088\build.log for details. Retry with --force to force install it.

[oalders]

There was a change in v6.11 https://metacpan.org/release/OALDERS/LWP-Protocol-https-6.14/source/Changes#L18

Are you able to install 6.10?

[Brian2099]

Thank you for the very quick response. I was able to find version 6.10, download and then install it. My Perl script is working now! Thank you very much!!! I hope someone provides a fix for the certificate issue before this module becomes too far out of date.

[oalders]

Are you able to install the latest version if you upgrade your Mozilla::CA?

[Brian2099]

I checked and see that I am using the latest version of Firefox – #127 From: Olaf Alders ***@***.***> Sent: Friday, June 28, 2024 5:40 PM To: libwww-perl/LWP-Protocol-https ***@***.***> Cc: Brian Ulmer ***@***.***>; Author ***@***.***> Subject: [EXTERNAL]: Re: [libwww-perl/LWP-Protocol-https] Install fails with certificate verify failed (Issue #80) CAUTION: This email originated from outside of the Federal Communications Commission. Do not click on links or open attachments unless you recognize the sender and trust the content to be safe. If you suspect this is a phishing attempt, please use the 'Report Message' feature in Microsoft Outlook or forward the email to the NSOC. Are you able to install the latest version if you upgrade your Mozilla::CA? — Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_libwww-2Dperl_LWP-2DProtocol-2Dhttps_issues_80-23issuecomment-2D2197694220&d=DwMFaQ&c=y0h0omCe0jAUGr4gAQ02Fw&r=mE7ADx4GP-4OaK3o1VILOkiqEl8-npXWakx9Szdzzv4&m=EhzFT1Z0eP3EoTJbIHjHs2Zt0WlCTqC0PHqCpg5leW_DZuzaQ_e-8ycAs1hqyKJV&s=xULj1psFJQgv5-oycXKX3JDaQ5veGCcDKD9bcldp_9U&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AANGSHF4WUTU5MYFF6TWGL3ZJXJZTAVCNFSM6AAAAABKCMH4E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJXGY4TIMRSGA&d=DwMFaQ&c=y0h0omCe0jAUGr4gAQ02Fw&r=mE7ADx4GP-4OaK3o1VILOkiqEl8-npXWakx9Szdzzv4&m=EhzFT1Z0eP3EoTJbIHjHs2Zt0WlCTqC0PHqCpg5leW_DZuzaQ_e-8ycAs1hqyKJV&s=kwXWj8-Jpx77-tYUW1yyn0eYgWMkEju_NuXZDQEk5lE&e=>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[karenetheridge]

I checked and see that I am using the latest version of Firefox – #127

He meant the cpan module -- https://metacpan.org/pod/Mozilla::CA

[Brian2099]

Karen, Thank you for the clarification. Mozilla::CA was out of date, so I did that install and followed up with another install of LWP::Protocol::https. Here are the results. The subsequent install of LWP::Protocol::https fails with slightly different errors. Any information on how to proceed would be appreciated. Thank you, Brian C:\myStuff\Perl>cpanm Mozilla::CA --> Working on Mozilla::CA Fetching http://www.cpan.org/authors/id/L/LW/LWP/Mozilla-CA-20240313.tar.gz ... OK Configuring Mozilla-CA-20240313 ... OK Building and testing Mozilla-CA-20240313 ... OK Successfully installed Mozilla-CA-20240313 (upgraded from 20230821) 1 distribution installed C:\myStuff\Perl>cpanm LWP::Protocol::https --> Working on LWP::Protocol::https Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/LWP-Protocol-https-6.14.tar.gz ... OK Configuring LWP-Protocol-https-6.14 ... OK Building and testing LWP-Protocol-https-6.14 ... FAIL ! Installing LWP::Protocol::https failed. See C:\Users\BRIAN~1\.cpanm\work\1719848586.30096\build.log for details. Retry with --force to force install it. C:\myStuff\Zendesk\Perl> Here is some information from the build.log file pertaining to the errors: # Failed test 'success status' # at t/example.t line 19. # Failed test 'have header Client-SSL-Socket-Class' # at t/example.t line 29. # Failed test 'have header Client-SSL-Cipher' # at t/example.t line 44. # Failed test 'found expected document content' # at t/example.t line 46. # 'Can't connect to www.example.com:443<http://www.example.com:443> (certificate verify failed) # # SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at C:/Strawberry/perl/vendor/lib/LWP/Protocol/http.pm line 50. # ' # doesn't match '(?^:Example Domain)' # Looks like you failed 4 tests of 6. From: Karen Etheridge ***@***.***> Sent: Monday, July 1, 2024 11:37 AM To: libwww-perl/LWP-Protocol-https ***@***.***> Cc: Brian Ulmer ***@***.***>; Author ***@***.***> Subject: [EXTERNAL]: Re: [libwww-perl/LWP-Protocol-https] Install fails with certificate verify failed (Issue #80) CAUTION: This email originated from outside of the Federal Communications Commission. Do not click on links or open attachments unless you recognize the sender and trust the content to be safe. If you suspect this is a phishing attempt, please use the 'Report Message' feature in Microsoft Outlook or forward the email to the NSOC. I checked and see that I am using the latest version of Firefox – #127 He meant the cpan module -- https://metacpan.org/pod/Mozilla::CA<https://urldefense.proofpoint.com/v2/url?u=https-3A__metacpan.org_pod_Mozilla-3A-3ACA&d=DwMFaQ&c=y0h0omCe0jAUGr4gAQ02Fw&r=mE7ADx4GP-4OaK3o1VILOkiqEl8-npXWakx9Szdzzv4&m=wTEnN-d5QJdEET0w5BRPr5wPyhQgNIt1ForQPpwJSmW2s5hIEGTQFyyocYmk0yHt&s=X-tsuTbLJGQuNyrQAbAVQ9-DwLqM855Q6-qq4xIBDJQ&e=> — Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_libwww-2Dperl_LWP-2DProtocol-2Dhttps_issues_80-23issuecomment-2D2200478137&d=DwMFaQ&c=y0h0omCe0jAUGr4gAQ02Fw&r=mE7ADx4GP-4OaK3o1VILOkiqEl8-npXWakx9Szdzzv4&m=wTEnN-d5QJdEET0w5BRPr5wPyhQgNIt1ForQPpwJSmW2s5hIEGTQFyyocYmk0yHt&s=yPphj2v8tNzyR1k--aokep1EIKktiBB2kvvpIiiHTwc&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AANGSHEX2Z3TBUUBAIKW4U3ZKFZQFAVCNFSM6AAAAABKCMH4E6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBQGQ3TQMJTG4&d=DwMFaQ&c=y0h0omCe0jAUGr4gAQ02Fw&r=mE7ADx4GP-4OaK3o1VILOkiqEl8-npXWakx9Szdzzv4&m=wTEnN-d5QJdEET0w5BRPr5wPyhQgNIt1ForQPpwJSmW2s5hIEGTQFyyocYmk0yHt&s=zNLr5BB5ojgm0mMlu3C9McMlctwwFIMxRqDRCS131KM&e=>. You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>

[Brian2099]

FYI, I also checked and see that I have the most current versions of IO::Socket::SSL and Crypt::SSLeay

[oalders]

@noxxi, any idea what might be going on here?

[eserte]

This test started to fail in the same way on some of my smokers (it seems fedora and rocky linux 9 is affected):

#   Failed test 'success status'
    #   at t/example.t line 19.

    #   Failed test 'have header Client-SSL-Socket-Class'
    #   at t/example.t line 29.

    #   Failed test 'have header Client-SSL-Cipher'
    #   at t/example.t line 44.

    #   Failed test 'found expected document content'
    #   at t/example.t line 46.
    #                   'Can't connect to www.example.com:443 (certificate verify failed)
    # 
    # SSL connect attempt failed error:0A000086:SSL routines::certificate verify failed at C:/Strawberry/perl/vendor/lib/LWP/Protocol/http.pm line 49.
    # '
    #     doesn't match '(?^:Example Domain)'
    # Looks like you failed 4 tests of 6.

#   Failed test 'Request GET https://www.example.com/'
#   at t/example.t line 47.

    #   Failed test 'success status'
    #   at t/example.t line 54.
    # Looks like you failed 1 test of 2.

#   Failed test 'Check for warnings from GET https://www.example.com/ (RT #81948)'
#   at t/example.t line 57.
# Looks like you failed 2 tests of 2.
t/example.t ............ 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/2 subtests

Looking at the www.example.com certificate it seems that it was renewed yesterday:

$ ssl-cert-validity www.example.com
notBefore=2026-02-13T18:53:48Z
notAfter=2026-05-14T18:57:50Z

Here's an analysis of fail reports against reported OpenSSL version (theta values towards zero means increasing amount of fail reports):

****************************************************************
Regression 'qr:stdout: OpenSSL (\S+)'
****************************************************************
Name                   Theta          StdErr     T-stat
[0='const']          -0.0000          0.1394      -0.00
[1='eq_1.0.1t']       0.4286          0.1581       2.71
[2='eq_1.0.2j']       0.9583          0.1451       6.60
[3='eq_1.0.2k-fips']          1.0000          0.1707       5.86
[4='eq_1.1.0l']       1.0000          0.1449       6.90
[5='eq_1.1.1n']       1.0000          0.1707       5.86
[6='eq_1.1.1w']       1.0000          0.1484       6.74
[7='eq_1.1.1w-freebsd']       1.0000          0.1541       6.49
[8='eq_3.0.13']       1.0000          0.1479       6.76
[9='eq_3.0.14']       1.0000          0.1649       6.06
[10='eq_3.0.15']              1.0000          0.1559       6.42
[11='eq_3.0.16']              1.0000          0.1506       6.64
[12='eq_3.0.18']              1.0000          0.2415       4.14
[13='eq_3.0.2']       1.0000          0.1541       6.49
[14='eq_3.0.7']       1.0000          0.1707       5.86
[15='eq_3.1.1']       1.0000          0.1516       6.60
[16='eq_3.2.2']       1.0000          0.1581       6.33
[17='eq_3.2.4']       1.0000          0.1541       6.49
[18='eq_3.3.0']       0.0000          0.2415       0.00
[19='eq_3.5.1']       0.2857          0.1581       1.81
[20='eq_3.5.4']       0.6364          0.1516       4.20
[21='eq_3.5.5']       1.0000          0.1972       5.07
[22='eq_3.6.0']       1.0000          0.2415       4.14

R^2= 0.571, N= 194, K= 23
****************************************************************

I also run the test suite on GitHub:
https://github.com/eserte/ci-helper-cpan-pm/actions/runs/22022319885/job/63633365872
It seems that also Windows and OpenBSD are affected.

[eserte]

It seems that adding SSL_verifiy_mode => 0 fixes the problem (tested on the fedora 43 system):

diff --git a/t/example.t b/t/example.t
index 16eae41..dfad9d7 100644
--- a/t/example.t
+++ b/t/example.t
@@ -6,7 +6,7 @@ use Test::RequiresInternet 'www.example.com' => 443;
 
 use LWP::UserAgent ();
 
-my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 } );
+my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0, SSL_verify_mode => 0 } );
 
 plan tests => 2;
 

[noxxi]

This test started to fail in the same way on some of my smokers (it seems fedora and rocky linux 9 is affected):

This is an issue with the underlying OS and of Mozilla::CA, not with LWP.

Looks like these systems (RockyLinux, Fedora, OpenBSD) and packages (Mozilla::CA) have removed the Comodo "AAA Certifica­te Services" root certificate, probably because it is signed with SHA-1 even if the signature algorithms on a self-signed root certificate is not relevant for validating the trust chain but only the contained public key. Unfortunately Cloudflare is using this root certificate as the base for their own intermediate certificate (with "SSL.com TLS Transit ECC CA R2" as an intermediate in between "AAA Certificates Services" and "Cloudflare TLS Issuing ECC CA 3") and the certificate for www.example.com is provided by Cloudflare.
Note that on these systems one does not even need Perl to show the problem - a simple openssl s_client or curl results in the same issue.

Using Mozilla::CA as trusted store instead of OS native does not help either, since the latest version (20250602) has removed this root CA too, probably because of libwww-perl/Mozilla-CA#39. That's why it fails on Windows too, since Mozilla::CA is used as trust store here.

Note that Mozilla has their own SHA-1 policy, which says nothing about SHA-1 signed root certificates but only sha-1 leaf and intermediate certificates - https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#513-sha-1.

And note that browsers seem to have a different certificate for "SSL.com TLS Transit ECC CA R2" built into their trust store, which does not chain to "AAA Certificates Services" as the one served by Cloudflare but instead chains to "SSL.com TLS ECC Root CA 2022" intermediate which then terminates in "SSL.com TLS ECC Root CA 2022", thus avoiding the old SHA-1 signed root CA of "AAA Certifica­te Services" . While the new root CA is contained in both Mozilla::CA and the affected OS, the new intermediate leading to the root is not, which means that the alternative trust path to the new root will not be found.