I have a VPS running C7. I ran a 'yum update' which included a kernel update. The yum update wasn't a total success because mariadb updates failed. However everything else appeared to work.
However, when I rebooted the server it did not restart. I have managed to get the ISP's support desk to boot the server by selecting the previous kernel, and I now have access to my box again.
As I will probably only have one chance to fix this (without having to get the ISP's help again) I was wondering if there are any clear instructions on how to remove a failed kernel RPM update, returning the server to the state it was in prior to the YUM update?
I have managed to find out what happened in the yum update and it turns out it was a mess. It looks like the server ran out of memory in the middle and things then started to fail. Any advice on how to recover from this would be greatly appreciated
The log below shows:
[root@vps2 ~]# yum history info 22 Loaded plugins: fastestmirror, langpacks Repository mariadb is listed more than once in the configuration Transaction ID : 22 Begin time : Sun Jan 19 18:15:33 2020 Begin rpmdb : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1 End time : 19:19:32 2020 (63 minutes) End rpmdb : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b User : root <root> Return-Code : Failure: 1 Command Line : update Transaction performed with: Installed rpm-4.11.3-40.el7.x86_64 @anaconda Installed yum-3.4.3-163.el7.centos.noarch @anaconda Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda Packages Altered: Updated MariaDB-client-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-common-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64 @mariadb Loading mirror speeds from cached hostfile * base: mirror.as29550.net * epel: www.mirrorservice.org * extras: www.mirrorservice.org * remi-php73: mirrors.ukfast.co.uk * remi-safe: mirrors.ukfast.co.uk * updates: www.mirrorservice.org ** Update 10.4.11-1.el7.centos.x86_64 @?mariadb ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch @anaconda ** Update 2019.2.32-76.el7_7.noarch @?updates Updated certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated clamav-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-data-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-devel-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-filesystem-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-lib-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-update-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamd-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated curl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated fail2ban-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-firewalld-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-sendmail-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-server-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-systemd-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fribidi-1.0.2-1.el7.x86_64 @anaconda Update 1.0.2-1.el7_7.1.x86_64 @updates Updated iproute-4.11.0-25.el7.x86_64 @anaconda Update 4.11.0-25.el7_7.2.x86_64 @updates Install kernel-3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated libblkid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libcurl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated libgs-9.25-2.el7_7.2.x86_64 @updates Update 9.25-2.el7_7.3.x86_64 @updates Updated libmount-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsmartcols-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libsss_nss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libuuid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libwebp7-1.0.2-1.el7.remi.x86_64 @remi-safe Update 1.0.3-1.el7.remi.x86_64 @remi-safe ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64 @updates Update 2:2.1-53.7.el7_7.x86_64 @updates Updated nss-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-softokn-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-softokn-freebl-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-sysinit-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-tools-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-util-3.44.0-3.el7.x86_64 @anaconda Update 3.44.0-4.el7_7.x86_64 @updates Updated numactl-libs-2.0.12-3.el7.x86_64 @anaconda Update 2.0.12-3.el7_7.1.x86_64 @updates Updated pgdg-redhat-repo-42.0-5.noarch @/pgdg-centos10-10-2.noarch Update 42.0-6.noarch @pgdg10 Updated php-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-cli-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-common-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-gd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-json-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-ldap-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-odbc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-pdo-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 ** Updated php-pear-1:1.10.10-2.el7.remi.noarch @remi-php70 ** Update 1:1.10.10-4.el7.remi.noarch @?remi-php73 Updated php-pgsql-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-process-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xml-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated postgresql10-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated proftpd-1.3.5e-7.el7.x86_64 @epel Update 1.3.5e-8.el7.x86_64 @epel Updated python-perf-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated python-requests-2.6.0-7.el7_7.noarch @updates Update 2.6.0-8.el7_7.noarch @updates Updated python2-acme-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-apache-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated selinux-policy-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated sos-3.7-7.el7.centos.noarch @updates Update 3.7-10.el7.centos.noarch @updates ** Updated sssd-client-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated tcpdump-14:4.9.2-4.el7.x86_64 @anaconda Update 14:4.9.2-4.el7_7.1.x86_64 @updates Updated util-linux-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Scriptlet output: 1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 2 error: Couldn't fork %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch): Cannot allocate memory 4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed 5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot allocate memory 6 error: Couldn't fork %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate memory 7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed 8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch): Cannot allocate memory 11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed 12 warning: /etc/clamd.d/scan.conf created as /etc/clamd.d/scan.conf.rpmnew 13 error: Couldn't fork %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory 14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped 15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped 16 error: Couldn't fork %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed 18 error: Couldn't fork %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped 20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64): Cannot allocate memory 21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed 22 error: Couldn't fork %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed 24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot allocate memory 25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot allocate memory 26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed 27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot allocate memory 28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot allocate memory 30 error: Couldn't fork %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory 31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot allocate memory 32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot allocate memory 33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot allocate memory 34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot allocate memory 35 error: Couldn't fork %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot allocate memory 37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64): Cannot allocate memory 40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64): Cannot allocate memory 41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot allocate memory 42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot allocate memory 43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed 44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64): Cannot allocate memory 45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 46 error: Couldn't fork %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory history info [root@vps2 ~]#
I have managed to find out what happened in the yum update and it turns out it was a mess. It looks like the server ran out of memory in the middle and things then started to fail. Any advice on how to recover from this would be greatly appreciated
I may sound old school but my suggestion would be to forget yum aka dnf for now and just fix the real stuff using rpm.
I always considered yum a wrapper for rpm (I wish it was) and was able to fix a system with rpm alone and later make yum behave again.
It usually boils down to analyzing what is in the rpm db and what is on the filesystem actually. You can then remove invalid rpm db content as well as wrong filesystem content. It's a bit of work and you may create helper scripts as needed but you can fix almost everything that way.
Regards, Simon
The log below shows:
[root@vps2 ~]# yum history info 22 Loaded plugins: fastestmirror, langpacks Repository mariadb is listed more than once in the configuration Transaction ID : 22 Begin time : Sun Jan 19 18:15:33 2020 Begin rpmdb : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1 End time : 19:19:32 2020 (63 minutes) End rpmdb : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b User : root <root> Return-Code : Failure: 1 Command Line : update Transaction performed with: Installed rpm-4.11.3-40.el7.x86_64 @anaconda Installed yum-3.4.3-163.el7.centos.noarch @anaconda Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda Packages Altered: Updated MariaDB-client-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-common-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64 @mariadb Loading mirror speeds from cached hostfile
- base: mirror.as29550.net
- epel: www.mirrorservice.org
- extras: www.mirrorservice.org
- remi-php73: mirrors.ukfast.co.uk
- remi-safe: mirrors.ukfast.co.uk
- updates: www.mirrorservice.org
** Update 10.4.11-1.el7.centos.x86_64 @?mariadb ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch @anaconda ** Update 2019.2.32-76.el7_7.noarch @?updates Updated certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated clamav-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-data-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-devel-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-filesystem-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-lib-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-update-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamd-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated curl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated fail2ban-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-firewalld-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-sendmail-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-server-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-systemd-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fribidi-1.0.2-1.el7.x86_64 @anaconda Update 1.0.2-1.el7_7.1.x86_64 @updates Updated iproute-4.11.0-25.el7.x86_64 @anaconda Update 4.11.0-25.el7_7.2.x86_64 @updates Install kernel-3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated libblkid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libcurl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated libgs-9.25-2.el7_7.2.x86_64 @updates Update 9.25-2.el7_7.3.x86_64 @updates Updated libmount-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsmartcols-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libsss_nss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libuuid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libwebp7-1.0.2-1.el7.remi.x86_64 @remi-safe Update 1.0.3-1.el7.remi.x86_64 @remi-safe ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64 @updates Update 2:2.1-53.7.el7_7.x86_64 @updates Updated nss-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-softokn-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-softokn-freebl-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-sysinit-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-tools-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-util-3.44.0-3.el7.x86_64 @anaconda Update 3.44.0-4.el7_7.x86_64 @updates Updated numactl-libs-2.0.12-3.el7.x86_64 @anaconda Update 2.0.12-3.el7_7.1.x86_64 @updates Updated pgdg-redhat-repo-42.0-5.noarch @/pgdg-centos10-10-2.noarch Update 42.0-6.noarch @pgdg10 Updated php-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-cli-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-common-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-gd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-json-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-ldap-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-odbc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-pdo-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 ** Updated php-pear-1:1.10.10-2.el7.remi.noarch @remi-php70 ** Update 1:1.10.10-4.el7.remi.noarch @?remi-php73 Updated php-pgsql-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-process-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xml-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated postgresql10-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated proftpd-1.3.5e-7.el7.x86_64 @epel Update 1.3.5e-8.el7.x86_64 @epel Updated python-perf-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated python-requests-2.6.0-7.el7_7.noarch @updates Update 2.6.0-8.el7_7.noarch @updates Updated python2-acme-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-apache-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated selinux-policy-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated sos-3.7-7.el7.centos.noarch @updates Update 3.7-10.el7.centos.noarch @updates ** Updated sssd-client-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated tcpdump-14:4.9.2-4.el7.x86_64 @anaconda Update 14:4.9.2-4.el7_7.1.x86_64 @updates Updated util-linux-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Scriptlet output: 1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 2 error: Couldn't fork %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch): Cannot allocate memory 4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed 5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot allocate memory 6 error: Couldn't fork %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate memory 7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed 8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch): Cannot allocate memory 11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed 12 warning: /etc/clamd.d/scan.conf created as /etc/clamd.d/scan.conf.rpmnew 13 error: Couldn't fork %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory 14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped 15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped 16 error: Couldn't fork %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed 18 error: Couldn't fork %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped 20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64): Cannot allocate memory 21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed 22 error: Couldn't fork %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed 24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot allocate memory 25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot allocate memory 26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed 27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot allocate memory 28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot allocate memory 30 error: Couldn't fork %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory 31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot allocate memory 32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot allocate memory 33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot allocate memory 34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot allocate memory 35 error: Couldn't fork %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot allocate memory 37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64): Cannot allocate memory 40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64): Cannot allocate memory 41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot allocate memory 42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot allocate memory 43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed 44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64): Cannot allocate memory 45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 46 error: Couldn't fork %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory history info [root@vps2 ~]# _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 1/22/20 3:57 PM, Gary Stainburn wrote:
I have managed to find out what happened in the yum update and it turns out it was a mess. It looks like the server ran out of memory in the middle and things then started to fail. Any advice on how to recover from this would be greatly appreciated
The log below shows:
[root@vps2 ~]# yum history info 22 Loaded plugins: fastestmirror, langpacks Repository mariadb is listed more than once in the configuration Transaction ID : 22 Begin time : Sun Jan 19 18:15:33 2020 Begin rpmdb : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1 End time : 19:19:32 2020 (63 minutes) End rpmdb : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b User : root <root> Return-Code : Failure: 1 Command Line : update Transaction performed with: Installed rpm-4.11.3-40.el7.x86_64 @anaconda Installed yum-3.4.3-163.el7.centos.noarch @anaconda Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda Packages Altered: Updated MariaDB-client-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-common-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64 @mariadb Loading mirror speeds from cached hostfile
- base: mirror.as29550.net
- epel: www.mirrorservice.org
- extras: www.mirrorservice.org
- remi-php73: mirrors.ukfast.co.uk
- remi-safe: mirrors.ukfast.co.uk
- updates: www.mirrorservice.org
** Update 10.4.11-1.el7.centos.x86_64 @?mariadb ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch @anaconda ** Update 2019.2.32-76.el7_7.noarch @?updates Updated certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated clamav-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-data-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-devel-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-filesystem-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-lib-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-update-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamd-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated curl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated fail2ban-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-firewalld-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-sendmail-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-server-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-systemd-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fribidi-1.0.2-1.el7.x86_64 @anaconda Update 1.0.2-1.el7_7.1.x86_64 @updates Updated iproute-4.11.0-25.el7.x86_64 @anaconda Update 4.11.0-25.el7_7.2.x86_64 @updates Install kernel-3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated libblkid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libcurl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated libgs-9.25-2.el7_7.2.x86_64 @updates Update 9.25-2.el7_7.3.x86_64 @updates Updated libmount-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsmartcols-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libsss_nss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libuuid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libwebp7-1.0.2-1.el7.remi.x86_64 @remi-safe Update 1.0.3-1.el7.remi.x86_64 @remi-safe ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64 @updates Update 2:2.1-53.7.el7_7.x86_64 @updates Updated nss-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-softokn-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-softokn-freebl-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-sysinit-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-tools-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-util-3.44.0-3.el7.x86_64 @anaconda Update 3.44.0-4.el7_7.x86_64 @updates Updated numactl-libs-2.0.12-3.el7.x86_64 @anaconda Update 2.0.12-3.el7_7.1.x86_64 @updates Updated pgdg-redhat-repo-42.0-5.noarch @/pgdg-centos10-10-2.noarch Update 42.0-6.noarch @pgdg10 Updated php-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-cli-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-common-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-gd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-json-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-ldap-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-odbc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-pdo-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 ** Updated php-pear-1:1.10.10-2.el7.remi.noarch @remi-php70 ** Update 1:1.10.10-4.el7.remi.noarch @?remi-php73 Updated php-pgsql-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-process-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xml-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated postgresql10-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated proftpd-1.3.5e-7.el7.x86_64 @epel Update 1.3.5e-8.el7.x86_64 @epel Updated python-perf-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated python-requests-2.6.0-7.el7_7.noarch @updates Update 2.6.0-8.el7_7.noarch @updates Updated python2-acme-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-apache-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated selinux-policy-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated sos-3.7-7.el7.centos.noarch @updates Update 3.7-10.el7.centos.noarch @updates ** Updated sssd-client-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated tcpdump-14:4.9.2-4.el7.x86_64 @anaconda Update 14:4.9.2-4.el7_7.1.x86_64 @updates Updated util-linux-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Scriptlet output: 1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 2 error: Couldn't fork %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch): Cannot allocate memory 4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed 5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot allocate memory 6 error: Couldn't fork %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate memory 7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed 8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch): Cannot allocate memory 11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed 12 warning: /etc/clamd.d/scan.conf created as /etc/clamd.d/scan.conf.rpmnew 13 error: Couldn't fork %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory 14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped 15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped 16 error: Couldn't fork %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed 18 error: Couldn't fork %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped 20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64): Cannot allocate memory 21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed 22 error: Couldn't fork %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed 24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot allocate memory 25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot allocate memory 26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed 27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot allocate memory 28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot allocate memory 30 error: Couldn't fork %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory 31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot allocate memory 32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot allocate memory 33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot allocate memory 34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot allocate memory 35 error: Couldn't fork %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot allocate memory 37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64): Cannot allocate memory 40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64): Cannot allocate memory 41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot allocate memory 42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot allocate memory 43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed 44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64): Cannot allocate memory 45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 46 error: Couldn't fork %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory history info [root@vps2 ~]#
Before you try the update again, you'll have to fix the reason for the failure - add memory, or at least add a swap file.
You could check with rpm -qa --last | head -20 which the latest packages are that were installed.
If the rpm database is corrupted, rebuild it with rpm --rebuilddb
You can reinstall packages that may not be completely installed, using rpm --reinstall PACKAGE_FILE
I'd rpm --erase the kernel package that does not boot.
HTH, Kay
On 1/22/20 3:57 PM, Gary Stainburn wrote:
I have managed to find out what happened in the yum update and it turns out it was a mess. It looks like the server ran out of memory in the middle and things then started to fail. Any advice on how to recover from this would be greatly appreciated
The log below shows:
[root@vps2 ~]# yum history info 22 Loaded plugins: fastestmirror, langpacks Repository mariadb is listed more than once in the configuration Transaction ID : 22 Begin time : Sun Jan 19 18:15:33 2020 Begin rpmdb : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1 End time : 19:19:32 2020 (63 minutes) End rpmdb : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b User : root <root> Return-Code : Failure: 1 Command Line : update Transaction performed with: Installed rpm-4.11.3-40.el7.x86_64 @anaconda Installed yum-3.4.3-163.el7.centos.noarch @anaconda Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda Packages Altered: Updated MariaDB-client-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-common-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64 @mariadb Loading mirror speeds from cached hostfile
- base: mirror.as29550.net
- epel: www.mirrorservice.org
- extras: www.mirrorservice.org
- remi-php73: mirrors.ukfast.co.uk
- remi-safe: mirrors.ukfast.co.uk
- updates: www.mirrorservice.org
** Update 10.4.11-1.el7.centos.x86_64 @?mariadb ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64 @mariadb Update 10.4.11-1.el7.centos.x86_64 @mariadb ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch @anaconda ** Update 2019.2.32-76.el7_7.noarch @?updates Updated certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated clamav-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-data-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-devel-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-filesystem-0.101.4-1.el7.noarch @epel Update 0.101.5-1.el7.noarch @epel Updated clamav-lib-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamav-update-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated clamd-0.101.4-1.el7.x86_64 @epel Update 0.101.5-1.el7.x86_64 @epel Updated curl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated fail2ban-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-firewalld-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-sendmail-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-server-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fail2ban-systemd-0.9.7-1.el7.noarch @epel Update 0.10.4-1.el7.noarch @epel Updated fribidi-1.0.2-1.el7.x86_64 @anaconda Update 1.0.2-1.el7_7.1.x86_64 @updates Updated iproute-4.11.0-25.el7.x86_64 @anaconda Update 4.11.0-25.el7_7.2.x86_64 @updates Install kernel-3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated libblkid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libcurl-7.29.0-54.el7.x86_64 @anaconda Update 7.29.0-54.el7_7.1.x86_64 @updates Updated libgs-9.25-2.el7_7.2.x86_64 @updates Update 9.25-2.el7_7.3.x86_64 @updates Updated libmount-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsmartcols-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libsss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libsss_nss_idmap-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated libuuid-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Updated libwebp7-1.0.2-1.el7.remi.x86_64 @remi-safe Update 1.0.3-1.el7.remi.x86_64 @remi-safe ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64 @updates Update 2:2.1-53.7.el7_7.x86_64 @updates Updated nss-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-softokn-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-softokn-freebl-3.44.0-5.el7.x86_64 @anaconda Update 3.44.0-8.el7_7.x86_64 @updates Updated nss-sysinit-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-tools-3.44.0-4.el7.x86_64 @anaconda Update 3.44.0-7.el7_7.x86_64 @updates Updated nss-util-3.44.0-3.el7.x86_64 @anaconda Update 3.44.0-4.el7_7.x86_64 @updates Updated numactl-libs-2.0.12-3.el7.x86_64 @anaconda Update 2.0.12-3.el7_7.1.x86_64 @updates Updated pgdg-redhat-repo-42.0-5.noarch @/pgdg-centos10-10-2.noarch Update 42.0-6.noarch @pgdg10 Updated php-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-cli-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-common-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-gd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-json-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-ldap-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-odbc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-pdo-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 ** Updated php-pear-1:1.10.10-2.el7.remi.noarch @remi-php70 ** Update 1:1.10.10-4.el7.remi.noarch @?remi-php73 Updated php-pgsql-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-process-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xml-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64 @remi-php73 Update 7.3.13-1.el7.remi.x86_64 @remi-php73 Updated postgresql10-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64 @pgdg10 Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 ** Updated proftpd-1.3.5e-7.el7.x86_64 @epel Update 1.3.5e-8.el7.x86_64 @epel Updated python-perf-3.10.0-1062.4.3.el7.x86_64 @updates Update 3.10.0-1062.9.1.el7.x86_64 @updates Updated python-requests-2.6.0-7.el7_7.noarch @updates Update 2.6.0-8.el7_7.noarch @updates Updated python2-acme-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated python2-certbot-apache-0.39.0-1.el7.noarch @epel Update 1.0.0-1.el7.noarch @epel Updated selinux-policy-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch @updates Update 3.13.1-252.el7_7.6.noarch @updates Updated sos-3.7-7.el7.centos.noarch @updates Update 3.7-10.el7.centos.noarch @updates ** Updated sssd-client-1.16.4-21.el7.x86_64 @anaconda Update 1.16.4-21.el7_7.1.x86_64 @updates Updated tcpdump-14:4.9.2-4.el7.x86_64 @anaconda Update 14:4.9.2-4.el7_7.1.x86_64 @updates Updated util-linux-2.23.2-61.el7.x86_64 @anaconda Update 2.23.2-61.el7_7.1.x86_64 @updates Scriptlet output: 1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 2 error: Couldn't fork %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch): Cannot allocate memory 4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed 5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot allocate memory 6 error: Couldn't fork %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate memory 7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed 8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory 9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch): Cannot allocate memory 11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed 12 warning: /etc/clamd.d/scan.conf created as /etc/clamd.d/scan.conf.rpmnew 13 error: Couldn't fork %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory 14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped 15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped 16 error: Couldn't fork %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed 18 error: Couldn't fork %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate memory 19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped 20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64): Cannot allocate memory 21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed 22 error: Couldn't fork %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed 24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot allocate memory 25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot allocate memory 26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed 27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot allocate memory 28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot allocate memory 30 error: Couldn't fork %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory 31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot allocate memory 32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot allocate memory 33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot allocate memory 34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot allocate memory 35 error: Couldn't fork %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory 36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot allocate memory 37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory 39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64): Cannot allocate memory 40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64): Cannot allocate memory 41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot allocate memory 42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot allocate memory 43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed 44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64): Cannot allocate memory 45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory 46 error: Couldn't fork %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory history info [root@vps2 ~]#
Before you try the update again, you'll have to fix the reason for the failure - add memory, or at least add a swap file.
You could check with rpm -qa --last | head -20 which the latest packages are that were installed.
If the rpm database is corrupted, rebuild it with rpm --rebuilddb
You can reinstall packages that may not be completely installed, using rpm --reinstall PACKAGE_FILE
Well, there may be more to check and above steps may not help.
Do you have duplicates in rpmdb? Which one of the duplicates are already on the filesystem?
Regards, Simon
On Thursday 23 January 2020 15:22:32 Simon Matter via CentOS wrote:
Before you try the update again, you'll have to fix the reason for the failure - add memory, or at least add a swap file.
You could check with rpm -qa --last | head -20 which the latest packages are that were installed.
If the rpm database is corrupted, rebuild it with rpm --rebuilddb
You can reinstall packages that may not be completely installed, using rpm --reinstall PACKAGE_FILE
Well, there may be more to check and above steps may not help.
Do you have duplicates in rpmdb? Which one of the duplicates are already on the filesystem?
I do not believe that I have any duplicate RPM's installed, and I do not believe that the database is actually wrong. I believe that it is purely that the installs did not complete successfully.
I have managed to remove the latest kernel RPM and 'yum update kernel*' re-applied the update without errors.
I am going to go through the list of failed RPM's and to a 'rpm --reinstall' for each one in turn and hopefully they will work too.
Before doing that I will do as Kay suggested and find out why I had the memory problem in the first place.
On 23/1/20 12:44 am, Gary Stainburn wrote:
... As I will probably only have one chance to fix this (without having to get the ISP's help again) I was wondering if there are any clear instructions on how to remove a failed kernel RPM update, returning the server to the state it was in prior to the YUM update? ...
If you do not want to involve the ISP ever again when a kernel won't boot, you can configure a fallback kernel [0]. I've used something similar a very long time ago when the main boot loader was LILO - it was much easier to do this kind of thing with it.
[0]: https://anindya.me/2011/09/17/grub-fallback-after-kernel-panic/
On 25/1/20 11:56 am, Anthony K wrote:
I've just discovered that the article I posted is a rather old post so went hunting.
If you have a RHEL subscription (I'm using the free developer subscription) you can see the official Red Hat documentation at [0]. I was unable to find similar instructions for CentOS but here's a post of the same vintage as the RHEL one [1].
[0]: https://access.redhat.com/solutions/1567883 [1]: https://forums.centos.org/viewtopic.php?t=60220
PS: I haven't tested either of these so no idea if they work as one would expect.
-
Anthony K -
Gary Stainburn -
Kay Diederichs -
Simon Matter