On 1/22/20 3:57 PM, Gary Stainburn wrote: > I have managed to find out what happened in the yum update and it turns out it was a mess. It looks like the server ran out of memory in the middle and things then started to fail. Any advice on how to recover from this would be greatly appreciated > > The log below shows: > > [root at vps2 ~]# yum history info 22 > Loaded plugins: fastestmirror, langpacks > Repository mariadb is listed more than once in the configuration > Transaction ID : 22 > Begin time : Sun Jan 19 18:15:33 2020 > Begin rpmdb : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1 > End time : 19:19:32 2020 (63 minutes) > End rpmdb : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b > User : root <root> > Return-Code : Failure: 1 > Command Line : update > Transaction performed with: > Installed rpm-4.11.3-40.el7.x86_64 @anaconda > Installed yum-3.4.3-163.el7.centos.noarch @anaconda > Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda > Packages Altered: > Updated MariaDB-client-10.4.10-1.el7.centos.x86_64 @mariadb > Update 10.4.11-1.el7.centos.x86_64 @mariadb > Updated MariaDB-common-10.4.10-1.el7.centos.x86_64 @mariadb > Update 10.4.11-1.el7.centos.x86_64 @mariadb > Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64 @mariadb > Update 10.4.11-1.el7.centos.x86_64 @mariadb > ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64 @mariadb > Loading mirror speeds from cached hostfile > * base: mirror.as29550.net > * epel: www.mirrorservice.org > * extras: www.mirrorservice.org > * remi-php73: mirrors.ukfast.co.uk > * remi-safe: mirrors.ukfast.co.uk > * updates: www.mirrorservice.org > ** Update 10.4.11-1.el7.centos.x86_64 @?mariadb > ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64 @mariadb > Update 10.4.11-1.el7.centos.x86_64 @mariadb > ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch @anaconda > ** Update 2019.2.32-76.el7_7.noarch @?updates > Updated certbot-0.39.0-1.el7.noarch @epel > Update 1.0.0-1.el7.noarch @epel > Updated clamav-0.101.4-1.el7.x86_64 @epel > Update 0.101.5-1.el7.x86_64 @epel > Updated clamav-data-0.101.4-1.el7.noarch @epel > Update 0.101.5-1.el7.noarch @epel > Updated clamav-devel-0.101.4-1.el7.x86_64 @epel > Update 0.101.5-1.el7.x86_64 @epel > Updated clamav-filesystem-0.101.4-1.el7.noarch @epel > Update 0.101.5-1.el7.noarch @epel > Updated clamav-lib-0.101.4-1.el7.x86_64 @epel > Update 0.101.5-1.el7.x86_64 @epel > Updated clamav-update-0.101.4-1.el7.x86_64 @epel > Update 0.101.5-1.el7.x86_64 @epel > Updated clamd-0.101.4-1.el7.x86_64 @epel > Update 0.101.5-1.el7.x86_64 @epel > Updated curl-7.29.0-54.el7.x86_64 @anaconda > Update 7.29.0-54.el7_7.1.x86_64 @updates > Updated fail2ban-0.9.7-1.el7.noarch @epel > Update 0.10.4-1.el7.noarch @epel > Updated fail2ban-firewalld-0.9.7-1.el7.noarch @epel > Update 0.10.4-1.el7.noarch @epel > Updated fail2ban-sendmail-0.9.7-1.el7.noarch @epel > Update 0.10.4-1.el7.noarch @epel > Updated fail2ban-server-0.9.7-1.el7.noarch @epel > Update 0.10.4-1.el7.noarch @epel > Updated fail2ban-systemd-0.9.7-1.el7.noarch @epel > Update 0.10.4-1.el7.noarch @epel > Updated fribidi-1.0.2-1.el7.x86_64 @anaconda > Update 1.0.2-1.el7_7.1.x86_64 @updates > Updated iproute-4.11.0-25.el7.x86_64 @anaconda > Update 4.11.0-25.el7_7.2.x86_64 @updates > Install kernel-3.10.0-1062.9.1.el7.x86_64 @updates > Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64 @updates > Update 3.10.0-1062.9.1.el7.x86_64 @updates > Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64 @updates > Update 3.10.0-1062.9.1.el7.x86_64 @updates > Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64 @updates > Update 3.10.0-1062.9.1.el7.x86_64 @updates > Updated libblkid-2.23.2-61.el7.x86_64 @anaconda > Update 2.23.2-61.el7_7.1.x86_64 @updates > Updated libcurl-7.29.0-54.el7.x86_64 @anaconda > Update 7.29.0-54.el7_7.1.x86_64 @updates > Updated libgs-9.25-2.el7_7.2.x86_64 @updates > Update 9.25-2.el7_7.3.x86_64 @updates > Updated libmount-2.23.2-61.el7.x86_64 @anaconda > Update 2.23.2-61.el7_7.1.x86_64 @updates > Updated libsmartcols-2.23.2-61.el7.x86_64 @anaconda > Update 2.23.2-61.el7_7.1.x86_64 @updates > Updated libsss_idmap-1.16.4-21.el7.x86_64 @anaconda > Update 1.16.4-21.el7_7.1.x86_64 @updates > Updated libsss_nss_idmap-1.16.4-21.el7.x86_64 @anaconda > Update 1.16.4-21.el7_7.1.x86_64 @updates > Updated libuuid-2.23.2-61.el7.x86_64 @anaconda > Update 2.23.2-61.el7_7.1.x86_64 @updates > Updated libwebp7-1.0.2-1.el7.remi.x86_64 @remi-safe > Update 1.0.3-1.el7.remi.x86_64 @remi-safe > ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64 @updates > Update 2:2.1-53.7.el7_7.x86_64 @updates > Updated nss-3.44.0-4.el7.x86_64 @anaconda > Update 3.44.0-7.el7_7.x86_64 @updates > Updated nss-softokn-3.44.0-5.el7.x86_64 @anaconda > Update 3.44.0-8.el7_7.x86_64 @updates > Updated nss-softokn-freebl-3.44.0-5.el7.x86_64 @anaconda > Update 3.44.0-8.el7_7.x86_64 @updates > Updated nss-sysinit-3.44.0-4.el7.x86_64 @anaconda > Update 3.44.0-7.el7_7.x86_64 @updates > Updated nss-tools-3.44.0-4.el7.x86_64 @anaconda > Update 3.44.0-7.el7_7.x86_64 @updates > Updated nss-util-3.44.0-3.el7.x86_64 @anaconda > Update 3.44.0-4.el7_7.x86_64 @updates > Updated numactl-libs-2.0.12-3.el7.x86_64 @anaconda > Update 2.0.12-3.el7_7.1.x86_64 @updates > Updated pgdg-redhat-repo-42.0-5.noarch @/pgdg-centos10-10-2.noarch > Update 42.0-6.noarch @pgdg10 > Updated php-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-cli-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-common-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-gd-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-json-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-ldap-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-odbc-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-pdo-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > ** Updated php-pear-1:1.10.10-2.el7.remi.noarch @remi-php70 > ** Update 1:1.10.10-4.el7.remi.noarch @?remi-php73 > Updated php-pgsql-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-process-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-xml-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64 @remi-php73 > Update 7.3.13-1.el7.remi.x86_64 @remi-php73 > Updated postgresql10-10.11-1PGDG.rhel7.x86_64 @pgdg10 > Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 > Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64 @pgdg10 > Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 > Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64 @pgdg10 > Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 > ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64 @pgdg10 > Update 10.11-2PGDG.rhel7.x86_64 @pgdg10 > ** Updated proftpd-1.3.5e-7.el7.x86_64 @epel > Update 1.3.5e-8.el7.x86_64 @epel > Updated python-perf-3.10.0-1062.4.3.el7.x86_64 @updates > Update 3.10.0-1062.9.1.el7.x86_64 @updates > Updated python-requests-2.6.0-7.el7_7.noarch @updates > Update 2.6.0-8.el7_7.noarch @updates > Updated python2-acme-0.39.0-1.el7.noarch @epel > Update 1.0.0-1.el7.noarch @epel > Updated python2-certbot-0.39.0-1.el7.noarch @epel > Update 1.0.0-1.el7.noarch @epel > Updated python2-certbot-apache-0.39.0-1.el7.noarch @epel > Update 1.0.0-1.el7.noarch @epel > Updated selinux-policy-3.13.1-252.el7.1.noarch @updates > Update 3.13.1-252.el7_7.6.noarch @updates > Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch @updates > Update 3.13.1-252.el7_7.6.noarch @updates > Updated sos-3.7-7.el7.centos.noarch @updates > Update 3.7-10.el7.centos.noarch @updates > ** Updated sssd-client-1.16.4-21.el7.x86_64 @anaconda > Update 1.16.4-21.el7_7.1.x86_64 @updates > Updated tcpdump-14:4.9.2-4.el7.x86_64 @anaconda > Update 14:4.9.2-4.el7_7.1.x86_64 @updates > Updated util-linux-2.23.2-61.el7.x86_64 @anaconda > Update 2.23.2-61.el7_7.1.x86_64 @updates > Scriptlet output: > 1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory > 2 error: Couldn't fork %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory > 3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch): Cannot allocate memory > 4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed > 5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot allocate memory > 6 error: Couldn't fork %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate memory > 7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed > 8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64): Cannot allocate memory > 9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory > 10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch): Cannot allocate memory > 11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed > 12 warning: /etc/clamd.d/scan.conf created as /etc/clamd.d/scan.conf.rpmnew > 13 error: Couldn't fork %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory > 14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped > 15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped > 16 error: Couldn't fork %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate memory > 17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed > 18 error: Couldn't fork %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate memory > 19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped > 20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64): Cannot allocate memory > 21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed > 22 error: Couldn't fork %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory > 23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed > 24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot allocate memory > 25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot allocate memory > 26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed > 27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot allocate memory > 28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory > 29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot allocate memory > 30 error: Couldn't fork %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory > 31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot allocate memory > 32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot allocate memory > 33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot allocate memory > 34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot allocate memory > 35 error: Couldn't fork %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate memory > 36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot allocate memory > 37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory > 38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64): Cannot allocate memory > 39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64): Cannot allocate memory > 40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64): Cannot allocate memory > 41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot allocate memory > 42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot allocate memory > 43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed > 44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64): Cannot allocate memory > 45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64): Cannot allocate memory > 46 error: Couldn't fork %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate memory > history info > [root at vps2 ~]# > Before you try the update again, you'll have to fix the reason for the failure - add memory, or at least add a swap file. You could check with rpm -qa --last | head -20 which the latest packages are that were installed. If the rpm database is corrupted, rebuild it with rpm --rebuilddb You can reinstall packages that may not be completely installed, using rpm --reinstall PACKAGE_FILE I'd rpm --erase the kernel package that does not boot. HTH, Kay