This evening I decided to do some work on my development C7 system. As I have not touched it for a while, and wanted to install new services I thought I'd better yum update first.
I saw that it only did updates from Google and PHP, and none from the system repo's so I had a closer look. It showed certificate errors on a number of repo's.
[root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: curl#60 - "Peer's Certificate issuer is not recognized." * base: mirrors.clouvider.net * epel: mirrors.coreix.net * extras: mirrors.clouvider.net * remi-php72: mirror.netweaver.uk * remi-safe: mirror.netweaver.uk * updates: mirrors.vooservers.com https://rpm.nodesource.com/pub_6.x/el/7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/.
https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/repoda...: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/repod...: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. No packages marked for update [root@stan2 ~]#
I wrongly followed the instructions in https://wiki.centos.org/yum-errors hoping that it would resolve the problems but it's only made it worse. Now all I get is:
[root@stan2 ~]# yum clean all Loaded plugins: fastestmirror, langpacks Cleaning repos: base epel extras google-chrome nodesource pgdg10 pgdg96 remi-php72 remi-safe updates Cleaning up list of fastest mirrors [root@stan2 ~]# rm -rf /var/cache/yum/* [root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors
One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid> or subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again [root@stan2 ~]#
I removed and re-installed the epel-release RPM which made no difference. I've also tried disabling each repo in turn, pgdg10, remi-php72, nodesource-el, pgdg96, google-chrome but still have no success.
Anyone got any suggestions?
On Aug 28, 2019, at 4:36 PM, Gary Stainburn gary.stainburn@ringways.co.uk wrote:
Anyone got any suggestions?
If it’s really out of date, you might need to update the ca-certificates package, but that’d have to be a really old system.
I’d suggest by checking to make sure the clock on your computer isn’t really out of date. If its right, I’d double-check with ‘curl’ to see if you aren’t getting a MitM response, where your HTTPS calls are being intercepted and resigned by a CA that isn’t in your CA trust. If that’s the case, you need be very suspicious of your network.
-- Jonathan Billings billings@negate.org
On Wednesday 28 August 2019 22:41:24 Jonathan Billings wrote:
If it’s really out of date, you might need to update the ca-certificates package, but that’d have to be a really old system.
I’d suggest by checking to make sure the clock on your computer isn’t really out of date. If its right, I’d double-check with ‘curl’ to see if you aren’t getting a MitM response, where your HTTPS calls are being intercepted and resigned by a CA that isn’t in your CA trust. If that’s the case, you need be very suspicious of your network.
It isn't that out of date. The server is less than a year old, and the last yum update was probably only done about 2 months ago.
I checked the system time and it was only a few minutes out. A quick rdate to my local time server sorted that.
I ran a yum check which took ages but didn't report any problems.
[root@stan2 ~]# yum check Loaded plugins: fastestmirror, langpacks check all [root@stan2 ~]#
However, running yum update afterwards came up with the same problem.
[root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors
One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid> or subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again [root@stan2 ~]# cat /etc/yum.repos.d/epel.repo [epel] name=Extra Packages for Enterprise Linux 7 - $basearch #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch failovermethod=priority enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
[epel-debuginfo] name=Extra Packages for Enterprise Linux 7 - $basearch - Debug #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basea... failovermethod=priority enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 gpgcheck=1
[epel-source] name=Extra Packages for Enterprise Linux 7 - $basearch - Source #baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$base... failovermethod=priority enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 gpgcheck=1 [root@stan2 ~]#
Interestingly, if I try a yum update on one of my other boxes I get similar errors. However, it then proceeds to complete the yum update successfully
[root@ollie2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: curl#60 - "Peer's Certificate issuer is not recognized." Could not retrieve mirrorlist https://mirror.webtatic.com/yum/el7/x86_64/mirrorlist error was 14: curl#60 - "Peer's Certificate issuer is not recognized." * base: mirror.as29550.net * epel: ftp-stud.hs-esslingen.de * extras: mozart.ee.ic.ac.uk * updates: mirror.vorboss.net * webtatic: uk.repo.webtatic.com base | 3.6 kB 00:00:00 http://download.owncloud.org/download/repositories/10.0/CentOS_7/repodata/re...: [Errno 14] HTTPS Error 302 - Found Trying other mirror. extras | 3.4 kB 00:00:00 google-chrome | 1.3 kB 00:00:00 https://rpm.nodesource.com/pub_6.x/el/7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/.
https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/repod...: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. updates | 3.4 kB 00:00:00 https://uk.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. (1/3): google-chrome/primary | 1.7 kB 00:00:00 (2/3): extras/7/x86_64/primary_db | 215 kB 00:00:00 (3/3): updates/7/x86_64/primary_db | 7.4 MB 00:00:05 google-chrome 3/3 Resolving Dependencies
On 8/29/19 3:03 AM, Gary Stainburn wrote:
https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized."
What do you see when you run:
openssl s_client -showcerts -connect us-east.repo.webtatic.com:443
On Thursday 29 August 2019 15:45:44 Gordon Messmer wrote:
On 8/29/19 3:03 AM, Gary Stainburn wrote:
https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized."
What do you see when you run:
openssl s_client -showcerts -connect us-east.repo.webtatic.com:443
That seems to work fine on the faulty server.
[root@stan2 ~]# openssl s_client -showcerts -connect us-east.repo.webtatic.com:443 CONNECTED(00000003) depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 verify return:1 depth=0 CN = webtatic.com verify return:1 --- Certificate chain 0 s:/CN=webtatic.com i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 -----BEGIN CERTIFICATE----- MIIF6jCCBNKgAwIBAgISBDXb5cfWLFXVBqOxkpcXwXVhMA0GCSqGSIb3DQEBCwUA MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xOTA3MTMyMjAwMTJaFw0x OTEwMTEyMjAwMTJaMBcxFTATBgNVBAMTDHdlYnRhdGljLmNvbTCCASIwDQYJKoZI hvcNAQEBBQADggEPADCCAQoCggEBAM3fbcrpxr9abHvq2fzpMhI1w5x03UZloW7u fPVx9qMQisH2rXYlaOi6JqvqutGemKuqeon97DmKNLC+uK7FNfhqm+M9bBiYYcp7 LEErsoTSpsG8+tACsuEEfI5VX668x+hVX9SRmt86qXS+ukvxiKGqaYyXc+9YonBU BUb1h24iiPP/U0wql6WpsZox6kaL4NDi53Fa6XzutNl7MO8SvWspRyccvOrFbSIa 60l2xQ3ZzwnBNE5PLjLNkaL/b/U5c6gAa+uDSpLGb5WLBVhXhtVM2nSxmR0WA+Mu GH7FDJZbXFoWh7Te7H6DVg64Muo2Cb9791zngJQcX835QpcKAecCAwEAAaOCAvsw ggL3MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH AwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU+yYwnaGc5M9ElauTeKw5gf9Uricw HwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEwbwYIKwYBBQUHAQEEYzBh MC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQub3Jn MC8GCCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMubGV0c2VuY3J5cHQub3Jn LzCBsAYDVR0RBIGoMIGlghNtaXJyb3Iud2VidGF0aWMuY29tghRubC5yZXBvLndl YnRhdGljLmNvbYIRcmVwby53ZWJ0YXRpYy5jb22CFHNwLnJlcG8ud2VidGF0aWMu Y29tghR1ay5yZXBvLndlYnRhdGljLmNvbYIZdXMtZWFzdC5yZXBvLndlYnRhdGlj LmNvbYIMd2VidGF0aWMuY29tghB3d3cud2VidGF0aWMuY29tMEwGA1UdIARFMEMw CAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9j cHMubGV0c2VuY3J5cHQub3JnMIIBBAYKKwYBBAHWeQIEAgSB9QSB8gDwAHYA4mlL ribo6UAJ6IYbtjuD1D7n/nSI+6SPKJMBnd3x2/4AAAFr7ZC2OwAABAMARzBFAiA2 oB+MtRoLHj2R10tZO68L/cCME2VGCM/WvwsbIAQz6wIhANmYApxOCCu4elrF+fMF b9BRooxn/wnAXgQNaXZMCTDJAHYAY/Lbzeg7zCzPC3KEJ1drM6SNYXePvXWmOLHH aFRL2I0AAAFr7ZC2LwAABAMARzBFAiAlxh9zfcwH3jblEejfwclCMCUcXYBUNBK4 tCFQ0lrQigIhAJL9l9eMgnWYuFgQcIHpfDhoPoR/1qUb7eulzCNEeuDHMA0GCSqG SIb3DQEBCwUAA4IBAQBy/d3y+sAM9iEE6pZkcbCONdbWeh8/g6o4VsFJ8c0K7MxR WAtiMgLK96SwhGHYrclvu9SMdi9B7umQtvxFRJq+jaFCANpddKcWegOlRwXhrMDs tOQhcMDnSZLJGjsFzwsYaluZlM1UI+xqnPR+fBoaLt3RaBQLowrsXpL4FMs+cJ0o /8ECkkIdZ2yJKzbt/XRc5Xj8cVo0lJXrZhqRJ3v0dJFLD4Sv+JQ9P91wlx8277Tk umcaa8fUOArtsaSxcnRkieJYainVv0b0YuZUZ1z0e94NPFAdY29hINBYfQQl6+wr zcQZke1Uc4S3edwPjZHX4M3KvEKFokRhlyaqSoTw -----END CERTIFICATE----- 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 i:/O=Digital Signature Trust Co./CN=DST Root CA X3 -----BEGIN CERTIFICATE----- MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/ MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8 SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0 Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj /PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/ wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6 KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg== -----END CERTIFICATE----- --- Server certificate subject=/CN=webtatic.com issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 --- No client certificate CA names sent Peer signing digest: SHA512 Server Temp Key: ECDH, P-256, 256 bits --- SSL handshake has read 3370 bytes and written 415 bytes --- New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: 3EAB66B1C00B5E9A0ABED3F0A58EBBB5EFCB92D21DA43C14947ACEA4740B5031 Session-ID-ctx: Master-Key: 45306EA81D8751DA376D3E0BDD15200AF59EC5F75B0FDFA7E6973469218E0EE947DABD22A1479A3076C903920C9DCB4A Key-Arg : None Krb5 Principal: None PSK identity: None PSK identity hint: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 7d 61 0c 5b 24 96 f1 c5-25 ef 1f e3 61 18 85 e0 }a.[$...%...a... 0010 - 81 71 15 64 13 a8 d3 f7-06 93 7e 91 d9 44 79 5e .q.d......~..Dy^ 0020 - 59 e5 6c 03 19 15 15 9a-28 e9 3b 3a 76 82 16 02 Y.l.....(.;:v... 0030 - 5b 1c 59 0e 15 08 99 48-02 65 f7 8d 46 76 f9 91 [.Y....H.e..Fv.. 0040 - e9 ea 08 b2 04 85 fc fc-49 60 f7 b2 a5 2d fc 09 ........I`...-.. 0050 - d3 4f 30 b7 98 5f 6c 3f-aa 5f eb aa d6 4c d1 a0 .O0.._l?._...L.. 0060 - 33 39 a1 fc 47 62 04 6b-58 2c 81 ca 84 e0 f3 c1 39..Gb.kX,...... 0070 - de 69 bb c1 91 5c d1 c2-bb 21 27 90 76 98 cb 11 .i......!'.v... 0080 - 4c cb 18 14 8c 91 0b 48-97 31 7c 31 32 c4 55 9e L......H.1|12.U. 0090 - a0 72 d6 ac 69 a3 2b b6-b7 61 8a 8d d9 e0 69 f1 .r..i.+..a....i. 00a0 - b5 2d 6c 66 12 04 f8 c0-65 05 21 27 1e 2c ec 1c .-lf....e.!'.,..
Start Time: 1567090119 Timeout : 300 (sec) Verify return code: 0 (ok) ---
DONE [root@stan2 ~]#
Am 2019-08-29 16:51, schrieb Gary Stainburn:
On Thursday 29 August 2019 15:45:44 Gordon Messmer wrote:
On 8/29/19 3:03 AM, Gary Stainburn wrote:
https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized."
What do you see when you run:
openssl s_client -showcerts -connect us-east.repo.webtatic.com:443
That seems to work fine on the faulty server.
[root@stan2 ~]# openssl s_client -showcerts -connect us-east.repo.webtatic.com:443 CONNECTED(00000003) depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 verify return:1 depth=0 CN = webtatic.com verify return:1
[ ... ]
Verify return code: 0 (ok)
Hi,
yum uses libcurl behind the scenes and thus NSS and not OpenSSL.
Do you get something indicative when running:
URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check-update
Alexander
On Thursday 29 August 2019 16:20:00 Alexander Dalloz wrote:
Hi,
yum uses libcurl behind the scenes and thus NSS and not OpenSSL.
Do you get something indicative when running:
URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check-update
Alexander
I get a lot of output for what looks like access to the local respos.d files all ending with success. I have included below the first and last of these immediately followed by the line saying that webtastic is not found
[root@stan2 ~]# URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check update 2019-08-29 16:27:08,367 urlgrabber version = 3.10 2019-08-29 16:27:08,367 trans function "_" = <function _ at 0x7effcd5bf2a8> 2019-08-29 16:27:08,391 combined options: { 'delegate' : { 'async' : None, 'bandwidth' : 0, 'cache_openers': True, 'checkfunc' : None, 'close_connection': 0, 'copy_local' : 0, 'curl_obj' : None, 'data' : None, 'default_speed': 1000000.0, 'delegate' : None, 'failfunc' : <function _do_raise at 0x7effcd5bf848>, 'failure_callback': None, 'ftp_disable_epsv': False, 'ftp_headers' : None, 'half_life' : 2592000, 'http_headers' : None, 'interrupt_callback': None, 'ip_resolve' : None, 'keepalive' : 1, 'libproxy' : False, 'max_connections': 5, 'max_header_size': 2097152, 'minrate' : None, 'mirror_group' : None, 'multi_progress_obj': None, 'no_cache' : False, 'opener' : None, 'password' : None, 'prefix' : None, 'progress_obj' : None, 'proxies' : None, 'proxy' : None, 'quote' : None, 'range' : None, 'reget' : None, 'retry' : None, 'retry_no_cache': False, 'retrycodes' : [-1, 2, 4, 5, 6, 7], 'size' : None, 'ssl_ca_cert' : None, 'ssl_cert' : None, 'ssl_cert_type': 'PEM', 'ssl_context' : None, 'ssl_key' : None, 'ssl_key_pass' : None, 'ssl_key_type' : 'PEM', 'ssl_verify_host': True, 'ssl_verify_peer': True, 'text' : None, 'throttle' : 1.0, 'timedhosts' : None, 'timeout' : 300, 'urlparser' : <urlgrabber.grabber.URLParser instance at 0x7effcd5bc518>, 'user_agent' : 'urlgrabber/3.10 yum/3.4.3', 'username' : None, } } 2019-08-29 16:27:08,392 attempt 1/None: file:///etc/yum.conf * Closing connection 0 2019-08-29 16:27:08,392 success
[big snip]
2019-08-29 16:27:08,460 combined options: { 'delegate' : { 'async' : None, 'bandwidth' : 0, 'cache_openers': True, 'checkfunc' : None, 'close_connection': 0, 'copy_local' : 0, 'curl_obj' : None, 'data' : None, 'default_speed': 1000000.0, 'delegate' : None, 'failfunc' : <function _do_raise at 0x7effcd5bf848>, 'failure_callback': None, 'ftp_disable_epsv': False, 'ftp_headers' : None, 'half_life' : 2592000, 'http_headers' : None, 'interrupt_callback': None, 'ip_resolve' : None, 'keepalive' : 1, 'libproxy' : False, 'max_connections': 5, 'max_header_size': 2097152, 'minrate' : None, 'mirror_group' : None, 'multi_progress_obj': None, 'no_cache' : False, 'opener' : None, 'password' : None, 'prefix' : None, 'progress_obj' : None, 'proxies' : None, 'proxy' : None, 'quote' : None, 'range' : None, 'reget' : None, 'retry' : None, 'retry_no_cache': False, 'retrycodes' : [-1, 2, 4, 5, 6, 7], 'size' : None, 'ssl_ca_cert' : None, 'ssl_cert' : None, 'ssl_cert_type': 'PEM', 'ssl_context' : None, 'ssl_key' : None, 'ssl_key_pass' : None, 'ssl_key_type' : 'PEM', 'ssl_verify_host': True, 'ssl_verify_peer': True, 'text' : None, 'throttle' : 1.0, 'timedhosts' : '/var/cache/yum/x86_64/7/timedhosts', 'timeout' : 300, 'urlparser' : <urlgrabber.grabber.URLParser instance at 0x7effcd5bc518>, 'user_agent' : 'urlgrabber/3.10 yum/3.4.3', 'username' : None, } } 2019-08-29 16:27:08,460 attempt 1/None: file:///etc/yum.repos.d/remi.repo * Closing connection 28 2019-08-29 16:27:08,460 success
Error getting repository data for webtatic, repository not found
Am 2019-08-29 17:36, schrieb Gary Stainburn:
On Thursday 29 August 2019 16:20:00 Alexander Dalloz wrote:
Hi,
yum uses libcurl behind the scenes and thus NSS and not OpenSSL.
Do you get something indicative when running:
URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check-update
Alexander
I get a lot of output for what looks like access to the local respos.d files all ending with success. I have included below the first and last of these immediately followed by the line saying that webtastic is not found
[root@stan2 ~]# URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check update
[ ... ]
Error getting repository data for webtatic, repository not found
Hm, I thought one of the repositories failing due to failing SSL is the webtatic one.
From your posting today 12:03 CEST:
* webtatic: uk.repo.webtatic.com
Anyhow, a test agaist "epel" would work too as it is configured to use a https target as well.
URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=epel update
Please check this too: rpm -Vv nss
Alexander
On Thursday 29 August 2019 16:47:11 Alexander Dalloz wrote:
rpm -Vv nss
[root@stan2 ~]# rpm -Vv nss ......... /etc/pki/nss-legacy ......... c /etc/pki/nss-legacy/nss-rhel7.config ......... /etc/pki/nssdb ......... c /etc/pki/nssdb/cert8.db ......... c /etc/pki/nssdb/cert9.db ......... c /etc/pki/nssdb/key3.db ......... c /etc/pki/nssdb/key4.db ......... c /etc/pki/nssdb/pkcs11.txt ......... c /etc/pki/nssdb/secmod.db ......... /usr/lib64/libnss3.so ......... g /usr/lib64/libnssckbi.so ......... /usr/lib64/libsmime3.so ......... /usr/lib64/libssl3.so ......... /usr/lib64/nss/libnssckbi.so ......... d /usr/share/man/man5/cert8.db.5.gz ......... d /usr/share/man/man5/cert9.db.5.gz ......... d /usr/share/man/man5/key3.db.5.gz ......... d /usr/share/man/man5/key4.db.5.gz ......... d /usr/share/man/man5/pkcs11.txt.5.gz ......... d /usr/share/man/man5/secmod.db.5.gz [root@stan2 ~]# URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=epel update [snip] Loading mirror speeds from cached hostfile 2019-08-29 17:23:17,344 combined options: { 'text' : 'epel/x86_64/metalink', 'delegate' : { 'async' : None, 'bandwidth' : 0, 'cache_openers': True, 'checkfunc' : None, 'close_connection': 0, 'copy_local' : 0, 'curl_obj' : None, 'data' : None, 'default_speed': 1000000.0, 'delegate' : None, 'failfunc' : <function _do_raise at 0x7fd3c45da848>, 'failure_callback': None, 'ftp_disable_epsv': False, 'ftp_headers' : None, 'half_life' : 2592000, 'http_headers' : (), 'interrupt_callback': None, 'ip_resolve' : None, 'keepalive' : True, 'libproxy' : False, 'max_connections': 5, 'max_header_size': 2097152, 'minrate' : 0, 'mirror_group' : None, 'multi_progress_obj': None, 'no_cache' : False, 'opener' : None, 'password' : None, 'prefix' : None, 'progress_obj' : None, 'proxies' : None, 'proxy' : None, 'quote' : None, 'range' : None, 'reget' : None, 'retry' : 10, 'retry_no_cache': False, 'retrycodes' : [-1, 2, 4, 5, 6, 7], 'size' : None, 'ssl_ca_cert' : None, 'ssl_cert' : None, 'ssl_cert_type': 'PEM', 'ssl_context' : None, 'ssl_key' : None, 'ssl_key_pass' : None, 'ssl_key_type' : 'PEM', 'ssl_verify_host': True, 'ssl_verify_peer': True, 'text' : None, 'throttle' : 0, 'timedhosts' : None, 'timeout' : 30.0, 'urlparser' : <urlgrabber.grabber.URLParser instance at 0x7fd3bf6f3ab8>, 'user_agent' : 'urlgrabber/3.10 yum/3.4.3', 'username' : None, } } 2019-08-29 17:23:17,344 attempt 1/10: https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 2019-08-29 17:23:17,345 opening local file "/var/cache/yum/x86_64/7/epel/metalink.xml.tmp" with mode wb * About to connect() to mirrors.fedoraproject.org port 443 (#29) * Trying 8.43.85.67... * Connected to mirrors.fedoraproject.org (8.43.85.67) port 443 (#29) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * Server certificate: * subject: CN=*.fedoraproject.org,O=Red Hat Inc.,L=Raleigh,ST=North Carolina,C=US * start date: Feb 01 00:00:00 2017 GMT * expire date: May 01 12:00:00 2020 GMT * common name: *.fedoraproject.org * issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US * NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER) * Peer's Certificate issuer is not recognized. * Closing connection 29 2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid> or subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
Am 2019-08-29 18:26, schrieb Gary Stainburn:
On Thursday 29 August 2019 16:47:11 Alexander Dalloz wrote:
rpm -Vv nss
[root@stan2 ~]# rpm -Vv nss ......... /etc/pki/nss-legacy ......... c /etc/pki/nss-legacy/nss-rhel7.config ......... /etc/pki/nssdb ......... c /etc/pki/nssdb/cert8.db ......... c /etc/pki/nssdb/cert9.db ......... c /etc/pki/nssdb/key3.db ......... c /etc/pki/nssdb/key4.db ......... c /etc/pki/nssdb/pkcs11.txt ......... c /etc/pki/nssdb/secmod.db ......... /usr/lib64/libnss3.so ......... g /usr/lib64/libnssckbi.so ......... /usr/lib64/libsmime3.so ......... /usr/lib64/libssl3.so ......... /usr/lib64/nss/libnssckbi.so ......... d /usr/share/man/man5/cert8.db.5.gz ......... d /usr/share/man/man5/cert9.db.5.gz ......... d /usr/share/man/man5/key3.db.5.gz ......... d /usr/share/man/man5/key4.db.5.gz ......... d /usr/share/man/man5/pkcs11.txt.5.gz ......... d /usr/share/man/man5/secmod.db.5.gz
Ok, that package content looks healthy. No problem there.
[root@stan2 ~]# URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=epel update [snip] Loading mirror speeds from cached hostfile 2019-08-29 17:23:17,344 combined options: { 'text' : 'epel/x86_64/metalink',
[ ... ]
2019-08-29 17:23:17,344 attempt 1/10: https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 2019-08-29 17:23:17,345 opening local file "/var/cache/yum/x86_64/7/epel/metalink.xml.tmp" with mode wb
- About to connect() to mirrors.fedoraproject.org port 443 (#29)
- Trying 8.43.85.67...
- Connected to mirrors.fedoraproject.org (8.43.85.67) port 443 (#29)
- Initializing NSS with certpath: sql:/etc/pki/nssdb
- CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none
- Server certificate:
- subject: CN=*.fedoraproject.org,O=Red Hat Inc.,L=Raleigh,ST=North
Carolina,C=US
- start date: Feb 01 00:00:00 2017 GMT
- expire date: May 01 12:00:00 2020 GMT
- common name: *.fedoraproject.org
- issuer: CN=DigiCert SHA2 High Assurance Server
CA,OU=www.digicert.com,O=DigiCert Inc,C=US
- NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER)
- Peer's Certificate issuer is not recognized.
So here we are.
While the current ca-certificates package of CentOS 7 ca-certificates-2018.2.22-70.0.el7_5.noarch does not hold the intermediate certificate "DigiCert SHA2 High Assurance Server" I don't get that issue.
# grep "DigiCert" /etc/pki/tls/certs/ca-bundle.crt # DigiCert Assured ID Root CA # DigiCert Assured ID Root G2 # DigiCert Assured ID Root G3 # DigiCert Global Root CA # DigiCert Global Root G2 # DigiCert Global Root G3 # DigiCert High Assurance EV Root CA # DigiCert Trusted Root G4
- Closing connection 29
2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
[ ... ]
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
So can we check what version of the ca-certificates packages is being installed on your system?
And a check into a different direction: what's the date and time of that system? Does it fit or is it wrong? Time being not accurate can make SSL connections fail.
Alexander
On Thursday 29 August 2019 18:10:19 Alexander Dalloz wrote:
2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
[ ... ]
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
So can we check what version of the ca-certificates packages is being installed on your system?
And a check into a different direction: what's the date and time of that system? Does it fit or is it wrong? Time being not accurate can make SSL connections fail.
Firstly, thank you for you help with this Alexander.
I had already checked the system time. It was about 3 minutes out, but I fixed it anyway. I have checked the RPM for the certificates, and it matches the one on another box that works.
[root@stan2 ~]# date Fri 30 Aug 09:45:27 BST 2019 [root@stan2 ~]# rpm -qa|grep cert ca-certificates-2018.2.22-70.0.el7_5.noarch [root@stan2 ~]#
In article 201908300952.37126.gary.stainburn@ringways.co.uk, Gary Stainburn gary.stainburn@ringways.co.uk wrote:
On Thursday 29 August 2019 18:10:19 Alexander Dalloz wrote:
2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
[ ... ]
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
So can we check what version of the ca-certificates packages is being installed on your system?
And a check into a different direction: what's the date and time of that system? Does it fit or is it wrong? Time being not accurate can make SSL connections fail.
Firstly, thank you for you help with this Alexander.
I had already checked the system time. It was about 3 minutes out, but I fixed it anyway. I have checked the RPM for the certificates, and it matches the one on another box that works.
[root@stan2 ~]# date Fri 30 Aug 09:45:27 BST 2019 [root@stan2 ~]# rpm -qa|grep cert ca-certificates-2018.2.22-70.0.el7_5.noarch [root@stan2 ~]#
Can you verify the ca-certificates package on both your systems and compare? Here is what my C7 box shows (same version package as yours):
[root@hp3 ~]# rpm -Vv ca-certificates ......... /etc/pki/ca-trust ......... /etc/pki/ca-trust/README ......... c /etc/pki/ca-trust/ca-legacy.conf ......... /etc/pki/ca-trust/extracted ......... /etc/pki/ca-trust/extracted/README ......... /etc/pki/ca-trust/extracted/java ......... /etc/pki/ca-trust/extracted/java/README .M....... g /etc/pki/ca-trust/extracted/java/cacerts ......... /etc/pki/ca-trust/extracted/openssl ......... /etc/pki/ca-trust/extracted/openssl/README .M....... g /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt ......... /etc/pki/ca-trust/extracted/pem ......... /etc/pki/ca-trust/extracted/pem/README .M....... g /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem .M....... g /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem .M....... g /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem ......... /etc/pki/ca-trust/source ......... /etc/pki/ca-trust/source/README ......... /etc/pki/ca-trust/source/anchors ......... /etc/pki/ca-trust/source/blacklist ......... g /etc/pki/ca-trust/source/ca-bundle.legacy.crt ......... /etc/pki/java ......... /etc/pki/java/cacerts ......... /etc/pki/tls ......... /etc/pki/tls/cert.pem ......... /etc/pki/tls/certs ......... /etc/pki/tls/certs/ca-bundle.crt ......... /etc/pki/tls/certs/ca-bundle.trust.crt ......... /etc/ssl ......... /etc/ssl/certs ......... /usr/bin/ca-legacy ......... /usr/bin/update-ca-trust ......... d /usr/share/doc/ca-certificates-2018.2.22/README ......... d /usr/share/man/man8/ca-legacy.8.gz ......... d /usr/share/man/man8/update-ca-trust.8.gz ......... /usr/share/pki ......... /usr/share/pki/ca-trust-legacy ......... /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.default.crt ......... /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.disable.crt ......... /usr/share/pki/ca-trust-source ......... /usr/share/pki/ca-trust-source/README ......... /usr/share/pki/ca-trust-source/anchors ......... /usr/share/pki/ca-trust-source/blacklist ......... /usr/share/pki/ca-trust-source/ca-bundle.trust.p11-kit [root@hp3 ~]#
And you could try re-installing ca-certificates on the offending box.
# yum --disablerepo=* --enablerepo=base --enablerepo=updates reinstall ca-certificates
Cheers Tony
On Friday 30 August 2019 11:51:35 Tony Mountifield wrote:
And you could try re-installing ca-certificates on the offending box.
# yum --disablerepo=* --enablerepo=base --enablerepo=updates reinstall ca-certificates
Cheers Tony
I have just done this and it appears to have fixed the problem. I have now successfully done a yum update including updates from the epel repo.
Thanks to everyone for your assistance.
Gary
Am 2019-08-30 10:52, schrieb Gary Stainburn:
On Thursday 29 August 2019 18:10:19 Alexander Dalloz wrote:
2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
[ ... ]
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
So can we check what version of the ca-certificates packages is being installed on your system?
And a check into a different direction: what's the date and time of that system? Does it fit or is it wrong? Time being not accurate can make SSL connections fail.
Firstly, thank you for you help with this Alexander.
You are welcome Gary. And I am curious about what the cause of your repo troubles is.
I had already checked the system time. It was about 3 minutes out, but I fixed it anyway. I have checked the RPM for the certificates, and it matches the one on another box that works.
[root@stan2 ~]# date Fri 30 Aug 09:45:27 BST 2019 [root@stan2 ~]# rpm -qa|grep cert ca-certificates-2018.2.22-70.0.el7_5.noarch [root@stan2 ~]#
That's good. Now please verify that the ca-certificates RPM is healthy:
rpm -V ca-certificates
In addition you can grep for the DigiCert certificates which are used by the fedoraproject.org mirror servers for EPEL (concentrating on a single broken HTTPS repo for now):
# grep "DigiCert" /etc/pki/tls/certs/ca-bundle.crt # DigiCert Assured ID Root CA # DigiCert Assured ID Root G2 # DigiCert Assured ID Root G3 # DigiCert Global Root CA # DigiCert Global Root G2 # DigiCert Global Root G3 # DigiCert High Assurance EV Root CA <<- that one must be there # DigiCert Trusted Root G4
Besides a corrupted certificates bundle I cannot imagine a different root cause actually.
Of course you could search system-wide for broken RPM content:
# for RPM in $(rpm -qa); do rpm -V ${RPM} >/dev/null; if [ "$?" -eq 1 ]; then echo "----- ${RPM} -----"; rpm -V ${RPM}; fi; done
Regards, Alexander
On Friday 30 August 2019 12:03:26 Alexander Dalloz wrote:
You are welcome Gary. And I am curious about what the cause of your repo troubles is.
I have looked back over what I have done, and cannot see what has caused the problem to occurr. I do not see anywhere where it could have been from any action that I have taken, including deleting the contents of the yum cache.
That's good. Now please verify that the ca-certificates RPM is healthy:
rpm -V ca-certificates
In addition you can grep for the DigiCert certificates which are used by the fedoraproject.org mirror servers for EPEL (concentrating on a single broken HTTPS repo for now):
# grep "DigiCert" /etc/pki/tls/certs/ca-bundle.crt # DigiCert Assured ID Root CA # DigiCert Assured ID Root G2 # DigiCert Assured ID Root G3 # DigiCert Global Root CA # DigiCert Global Root G2 # DigiCert Global Root G3 # DigiCert High Assurance EV Root CA <<- that one must be there # DigiCert Trusted Root G4
Besides a corrupted certificates bundle I cannot imagine a different root cause actually.
I have done both of these steps and got the same results as you. This may be because I have already re-installed ca-certificates as Tony's suggestion.
The main thing is that the server now has a working yum once again, but it wouls have been nice to find out the original cause.
Once again, thanks to everyone for you assistance. I found it very educational.
On Fri, Aug 30, 2019 at 12:17:47PM +0100, Gary Stainburn wrote:
On Friday 30 August 2019 12:03:26 Alexander Dalloz wrote:
Besides a corrupted certificates bundle I cannot imagine a different root cause actually.
Just to mention that the 'etckeeper' package from EPEL is great for tracking changes to /etc. Package installs trigger a commit, as do a daily cron job.
If in this case it was a corrupt file in /etc/pki, then a 'git log' or similar could show when it happened. Although I think you tried 'rpm -V' already so perhaps it wasn't a corrupt cert file.
Paddy
On Friday 30 August 2019 12:45:04 Paddy Doyle wrote:
Just to mention that the 'etckeeper' package from EPEL is great for tracking changes to /etc. Package installs trigger a commit, as do a daily cron job.
If in this case it was a corrupt file in /etc/pki, then a 'git log' or similar could show when it happened. Although I think you tried 'rpm -V' already so perhaps it wasn't a corrupt cert file.
Paddy
Hi Paddy,
Thanks for this. I'll have a look.
Incidentally, the *good* server that I was referencing my broken server against has decided to start giving the curl certificate errors in the same way that the broken one did. Very strange. I ran
yum --disablerepo=* --enablerepo=base --enablerepo=updates reinstall ca-certificates
on this server and again it fixed the problem. This would suggest that the problem is actually external to the original broken server.
On 8/30/19 5:52 AM, Gary Stainburn wrote:
Incidentally, the*good* server that I was referencing my broken server against has decided to start giving the curl certificate errors in the same way that the broken one did. Very strange. I ran
It's possible that the error is unrelated to the ca-certificates file. You'll only see it if yum selects a mirror that uses a Let's Encrypt or Amazon-signed certificate (at least, those were the CAs for the hosts I saw you report errors for). If yum happens to select mirrors that don't, then everything will work normally. Reinstalling the package on the original system may have been coincidental.
On Friday 30 August 2019 16:04:51 Gordon Messmer wrote:
On 8/30/19 5:52 AM, Gary Stainburn wrote:
Incidentally, the*good* server that I was referencing my broken server against has decided to start giving the curl certificate errors in the same way that the broken one did. Very strange. I ran
It's possible that the error is unrelated to the ca-certificates file. You'll only see it if yum selects a mirror that uses a Let's Encrypt or Amazon-signed certificate (at least, those were the CAs for the hosts I saw you report errors for). If yum happens to select mirrors that don't, then everything will work normally. Reinstalling the package on the original system may have been coincidental.
Hi Gordon,
That would make a great deal of sense, and fits in with the external influence which would explain why it's suddenly appearing on both servers.
However, when I re-installed ca-certificates it immediately fixed the problem on both boxes, which implies an internal problem.
Gary
On 8/30/19 8:17 AM, Gary Stainburn wrote:
However, when I re-installed ca-certificates it immediately fixed the problem on both boxes, which implies an internal problem.
That is only true if yum selected the same server, and there is no evidence that it did. It's possible that reinstalling the package fixed the problem, and it's also possible that it did not.
Am 2019-08-30 17:04, schrieb Gordon Messmer:
On 8/30/19 5:52 AM, Gary Stainburn wrote:
Incidentally, the*good* server that I was referencing my broken server against has decided to start giving the curl certificate errors in the same way that the broken one did. Very strange. I ran
It's possible that the error is unrelated to the ca-certificates file. You'll only see it if yum selects a mirror that uses a Let's Encrypt or Amazon-signed certificate (at least, those were the CAs for the hosts I saw you report errors for). If yum happens to select mirrors that don't, then everything will work normally. Reinstalling the package on the original system may have been coincidental.
Testing yum's activity in debug mode had shown:
https://lists.centos.org/pipermail/centos/2019-August/173297.html
2019-08-29 17:23:17,345 opening local file "/var/cache/yum/x86_64/7/epel/metalink.xml.tmp" with mode wb * About to connect() to mirrors.fedoraproject.org port 443 (#29) * Trying 8.43.85.67... * Connected to mirrors.fedoraproject.org (8.43.85.67) port 443 (#29) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * Server certificate: * subject: CN=*.fedoraproject.org,O=Red Hat Inc.,L=Raleigh,ST=North Carolina,C=US * start date: Feb 01 00:00:00 2017 GMT * expire date: May 01 12:00:00 2020 GMT * common name: *.fedoraproject.org * issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US * NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER) * Peer's Certificate issuer is not recognized. * Closing connection 29 2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." 2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising
Based on that it appears to me very clear that the trust with the DigiCert chain wasn't given due to a missing trust from the ca-cert bundle. Unfortunately we haven't seen a status of the ca-certificates RPM content before fixing it with a reinstall.
Alexander
On 8/29/19 8:20 AM, Alexander Dalloz wrote:
yum uses libcurl behind the scenes and thus NSS and not OpenSSL.
Good to know.
In that case: Gary, what do you see when you run:
/usr/lib64/nss/unsupported-tools/vfyserv -p 443 us-east.repo.webtatic.com
Do you get something indicative when running: URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check-update
webtatic is the mirror, not a repo. That won't do anything useful, will it?
Am 2019-08-30 17:17, schrieb Gordon Messmer:
On 8/29/19 8:20 AM, Alexander Dalloz wrote:
yum uses libcurl behind the scenes and thus NSS and not OpenSSL.
Good to know.
In that case: Gary, what do you see when you run:
/usr/lib64/nss/unsupported-tools/vfyserv -p 443 us-east.repo.webtatic.com
Do you get something indicative when running: URLGRABBER_DEBUG=1 yum --disablerepo=* --enablerepo=webtatic check-update
webtatic is the mirror, not a repo. That won't do anything useful, will it?
In posting https://lists.centos.org/pipermail/centos/2019-August/173288.html you could see that he has a repo "webtatic" configured, at that time calling a different mirror.
Alexander
On Friday 30 August 2019 16:27:01 Alexander Dalloz wrote:
In posting https://lists.centos.org/pipermail/centos/2019-August/173288.html you could see that he has a repo "webtatic" configured, at that time calling a different mirror.
Alexander
As far as I know I've never had webtatic as a repo. AFAIK it's a mirror used by remi (?).
However in the email you refer to it definitely appears in the repo list in the output from "yum update".
I don't understand that.
On 29/08/19 9:58 PM, Gary Stainburn wrote:
One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
I would try this:
yum clean all yum --disablerepo=epel update yum --disablerepo=epel --enablerepo=extras reinstall epel-release yum update
If that doesn't work show the complete output from the above commands and we'll go from there.
Peter
On Friday 30 August 2019 04:54:14 Peter wrote:
I would try this:
yum clean all
ran okay.
yum --disablerepo=epel update
ran okay but said there was nothing to update which I find hard to believe. It has been a month or so at least since the last successful update. It did complain about the REMI repos, which is odd as this all started when my yum update only updated PHP and Google Chrome, with PHP coming from REMI.
yum --disablerepo=epel --enablerepo=extras reinstall epel-release
ran okay and successfully reinstalled epel-release.noarch 0:7-11
yum update
Still failed in the same way as before. Full output below.
[root@stan2 ~]# yum clean all Loaded plugins: fastestmirror, langpacks Cleaning repos: base epel extras remi-php72 remi-safe updates Cleaning up list of fastest mirrors Other repos take up 57 k of disk space (use --verbose for details) [root@stan2 ~]# yum --disablerepo=epel update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors * base: mirror.sov.uk.goscomb.net * extras: mirror.clustered.net * remi-php72: mirror.23media.com * remi-safe: mirror.23media.com * updates: mirrors.vooservers.com base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 https://mirror.23media.com/remi/enterprise/7/php72/x86_64/repodata/repomd.xm...: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.centos.org/yum-errors If above article doesn't help to resolve this issue please use https://bugs.centos.org/.
https://mirror.oxilion.nl/remi/enterprise/7/php72/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." Trying other mirror. https://mirrors.ukfast.co.uk/sites/remi/enterprise/7/php72/x86_64/repodata/r...: [Errno 14] curl#60 - "Peer's Certificate issuer is not recognized." Trying other mirror. remi-php72 | 3.0 kB 00:00:00 remi-safe | 3.0 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/6): base/7/x86_64/group_gz | 166 kB 00:00:00 (2/6): extras/7/x86_64/primary_db | 215 kB 00:00:00 (3/6): remi-php72/primary_db | 225 kB 00:00:00 (4/6): remi-safe/primary_db | 1.6 MB 00:00:02 (5/6): updates/7/x86_64/primary_db | 7.4 MB 00:00:02 (6/6): base/7/x86_64/primary_db | 6.0 MB 00:00:03 No packages marked for update [root@stan2 ~]# yum --disablerepo=epel --enablerepo=extras reinstall epel-release Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirror.sov.uk.goscomb.net * extras: mirror.clustered.net * remi-php72: mirror.23media.com * remi-safe: mirror.23media.com * updates: mirrors.vooservers.com Resolving Dependencies --> Running transaction check ---> Package epel-release.noarch 0:7-11 will be reinstalled --> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Reinstalling: epel-release noarch 7-11 extras 15 k
Transaction Summary ============================================================================================================================================================== Reinstall 1 Package
Total download size: 15 k Installed size: 24 k Is this ok [y/d/N]: y Downloading packages: epel-release-7-11.noarch.rpm | 15 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum. Installing : epel-release-7-11.noarch 1/1 Verifying : epel-release-7-11.noarch 1/1
Installed: epel-release.noarch 0:7-11
Complete! [root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile
One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid> or subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again [root@stan2 ~]#
On 30/08/19 9:02 PM, Gary Stainburn wrote:
[root@stan2 ~]# yum update
2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
At this point I would follow the above advice, basically comment out the metalink= lines in your epel.repo file and uncomment the baseurl lines, then try the update again.
Later on you should try changing them back, but it appears as if for now at least you're having issues downloading the metalink file.
Peter
-
Alexander Dalloz -
Gary Stainburn -
Gordon Messmer -
Jonathan Billings -
Paddy Doyle -
Peter -
tony＠softins.co.uk