[CentOS] yum update / kernal update failed - remove or repair

Thu Jan 23 11:15:37 UTC 2020
Simon Matter <simon.matter at invoca.ch>

> I have managed to find out what happened in the yum update and it turns
> out it was a mess.  It looks like the server ran out of memory in the
> middle and things then started to fail.  Any advice on how to recover from
> this would be greatly appreciated

I may sound old school but my suggestion would be to forget yum aka dnf
for now and just fix the real stuff using rpm.

I always considered yum a wrapper for rpm (I wish it was) and was able to
fix a system with rpm alone and later make yum behave again.

It usually boils down to analyzing what is in the rpm db and what is on
the filesystem actually. You can then remove invalid rpm db content as
well as wrong filesystem content. It's a bit of work and you may create
helper scripts as needed but you can fix almost everything that way.

Regards,
Simon

>
> The log below shows:
>
> [root at vps2 ~]# yum history info 22
> Loaded plugins: fastestmirror, langpacks
> Repository mariadb is listed more than once in the configuration
> Transaction ID : 22
> Begin time     : Sun Jan 19 18:15:33 2020
> Begin rpmdb    : 795:6c5cd82e3ada047c7071fea1524f41fd13fab6a1
> End time       :            19:19:32 2020 (63 minutes)
> End rpmdb      : 801:6d1b278ccbeef6f68ff494cb28eaed5c7be4124b
> User           : root <root>
> Return-Code    : Failure: 1
> Command Line   : update
> Transaction performed with:
>     Installed     rpm-4.11.3-40.el7.x86_64                      @anaconda
>     Installed     yum-3.4.3-163.el7.centos.noarch               @anaconda
>     Installed     yum-plugin-fastestmirror-1.1.31-52.el7.noarch @anaconda
> Packages Altered:
>     Updated MariaDB-client-10.4.10-1.el7.centos.x86_64        @mariadb
>     Update                 10.4.11-1.el7.centos.x86_64        @mariadb
>     Updated MariaDB-common-10.4.10-1.el7.centos.x86_64        @mariadb
>     Update                 10.4.11-1.el7.centos.x86_64        @mariadb
>     Updated MariaDB-compat-10.4.10-1.el7.centos.x86_64        @mariadb
>     Update                 10.4.11-1.el7.centos.x86_64        @mariadb
>  ** Updated MariaDB-server-10.4.10-1.el7.centos.x86_64        @mariadb
> Loading mirror speeds from cached hostfile
>  * base: mirror.as29550.net
>  * epel: www.mirrorservice.org
>  * extras: www.mirrorservice.org
>  * remi-php73: mirrors.ukfast.co.uk
>  * remi-safe: mirrors.ukfast.co.uk
>  * updates: www.mirrorservice.org
>  ** Update                 10.4.11-1.el7.centos.x86_64        @?mariadb
>  ** Updated MariaDB-shared-10.4.10-1.el7.centos.x86_64        @mariadb
>     Update                 10.4.11-1.el7.centos.x86_64        @mariadb
>  ** Updated ca-certificates-2018.2.22-70.0.el7_5.noarch       @anaconda
>  ** Update                  2019.2.32-76.el7_7.noarch         @?updates
>     Updated certbot-0.39.0-1.el7.noarch                       @epel
>     Update          1.0.0-1.el7.noarch                        @epel
>     Updated clamav-0.101.4-1.el7.x86_64                       @epel
>     Update         0.101.5-1.el7.x86_64                       @epel
>     Updated clamav-data-0.101.4-1.el7.noarch                  @epel
>     Update              0.101.5-1.el7.noarch                  @epel
>     Updated clamav-devel-0.101.4-1.el7.x86_64                 @epel
>     Update               0.101.5-1.el7.x86_64                 @epel
>     Updated clamav-filesystem-0.101.4-1.el7.noarch            @epel
>     Update                    0.101.5-1.el7.noarch            @epel
>     Updated clamav-lib-0.101.4-1.el7.x86_64                   @epel
>     Update             0.101.5-1.el7.x86_64                   @epel
>     Updated clamav-update-0.101.4-1.el7.x86_64                @epel
>     Update                0.101.5-1.el7.x86_64                @epel
>     Updated clamd-0.101.4-1.el7.x86_64                        @epel
>     Update        0.101.5-1.el7.x86_64                        @epel
>     Updated curl-7.29.0-54.el7.x86_64                         @anaconda
>     Update       7.29.0-54.el7_7.1.x86_64                     @updates
>     Updated fail2ban-0.9.7-1.el7.noarch                       @epel
>     Update           0.10.4-1.el7.noarch                      @epel
>     Updated fail2ban-firewalld-0.9.7-1.el7.noarch             @epel
>     Update                     0.10.4-1.el7.noarch            @epel
>     Updated fail2ban-sendmail-0.9.7-1.el7.noarch              @epel
>     Update                    0.10.4-1.el7.noarch             @epel
>     Updated fail2ban-server-0.9.7-1.el7.noarch                @epel
>     Update                  0.10.4-1.el7.noarch               @epel
>     Updated fail2ban-systemd-0.9.7-1.el7.noarch               @epel
>     Update                   0.10.4-1.el7.noarch              @epel
>     Updated fribidi-1.0.2-1.el7.x86_64                        @anaconda
>     Update          1.0.2-1.el7_7.1.x86_64                    @updates
>     Updated iproute-4.11.0-25.el7.x86_64                      @anaconda
>     Update          4.11.0-25.el7_7.2.x86_64                  @updates
>     Install kernel-3.10.0-1062.9.1.el7.x86_64                 @updates
>     Updated kernel-headers-3.10.0-1062.4.3.el7.x86_64         @updates
>     Update                 3.10.0-1062.9.1.el7.x86_64         @updates
>     Updated kernel-tools-3.10.0-1062.4.3.el7.x86_64           @updates
>     Update               3.10.0-1062.9.1.el7.x86_64           @updates
>     Updated kernel-tools-libs-3.10.0-1062.4.3.el7.x86_64      @updates
>     Update                    3.10.0-1062.9.1.el7.x86_64      @updates
>     Updated libblkid-2.23.2-61.el7.x86_64                     @anaconda
>     Update           2.23.2-61.el7_7.1.x86_64                 @updates
>     Updated libcurl-7.29.0-54.el7.x86_64                      @anaconda
>     Update          7.29.0-54.el7_7.1.x86_64                  @updates
>     Updated libgs-9.25-2.el7_7.2.x86_64                       @updates
>     Update        9.25-2.el7_7.3.x86_64                       @updates
>     Updated libmount-2.23.2-61.el7.x86_64                     @anaconda
>     Update           2.23.2-61.el7_7.1.x86_64                 @updates
>     Updated libsmartcols-2.23.2-61.el7.x86_64                 @anaconda
>     Update               2.23.2-61.el7_7.1.x86_64             @updates
>     Updated libsss_idmap-1.16.4-21.el7.x86_64                 @anaconda
>     Update               1.16.4-21.el7_7.1.x86_64             @updates
>     Updated libsss_nss_idmap-1.16.4-21.el7.x86_64             @anaconda
>     Update                   1.16.4-21.el7_7.1.x86_64         @updates
>     Updated libuuid-2.23.2-61.el7.x86_64                      @anaconda
>     Update          2.23.2-61.el7_7.1.x86_64                  @updates
>     Updated libwebp7-1.0.2-1.el7.remi.x86_64                  @remi-safe
>     Update           1.0.3-1.el7.remi.x86_64                  @remi-safe
>  ** Updated microcode_ctl-2:2.1-53.3.el7_7.x86_64             @updates
>     Update                2:2.1-53.7.el7_7.x86_64             @updates
>     Updated nss-3.44.0-4.el7.x86_64                           @anaconda
>     Update      3.44.0-7.el7_7.x86_64                         @updates
>     Updated nss-softokn-3.44.0-5.el7.x86_64                   @anaconda
>     Update              3.44.0-8.el7_7.x86_64                 @updates
>     Updated nss-softokn-freebl-3.44.0-5.el7.x86_64            @anaconda
>     Update                     3.44.0-8.el7_7.x86_64          @updates
>     Updated nss-sysinit-3.44.0-4.el7.x86_64                   @anaconda
>     Update              3.44.0-7.el7_7.x86_64                 @updates
>     Updated nss-tools-3.44.0-4.el7.x86_64                     @anaconda
>     Update            3.44.0-7.el7_7.x86_64                   @updates
>     Updated nss-util-3.44.0-3.el7.x86_64                      @anaconda
>     Update           3.44.0-4.el7_7.x86_64                    @updates
>     Updated numactl-libs-2.0.12-3.el7.x86_64                  @anaconda
>     Update               2.0.12-3.el7_7.1.x86_64              @updates
>     Updated pgdg-redhat-repo-42.0-5.noarch
> @/pgdg-centos10-10-2.noarch
>     Update                   42.0-6.noarch                    @pgdg10
>     Updated php-7.3.12-1.el7.remi.x86_64                      @remi-php73
>     Update      7.3.13-1.el7.remi.x86_64                      @remi-php73
>     Updated php-cli-7.3.12-1.el7.remi.x86_64                  @remi-php73
>     Update          7.3.13-1.el7.remi.x86_64                  @remi-php73
>     Updated php-common-7.3.12-1.el7.remi.x86_64               @remi-php73
>     Update             7.3.13-1.el7.remi.x86_64               @remi-php73
>     Updated php-gd-7.3.12-1.el7.remi.x86_64                   @remi-php73
>     Update         7.3.13-1.el7.remi.x86_64                   @remi-php73
>     Updated php-json-7.3.12-1.el7.remi.x86_64                 @remi-php73
>     Update           7.3.13-1.el7.remi.x86_64                 @remi-php73
>     Updated php-ldap-7.3.12-1.el7.remi.x86_64                 @remi-php73
>     Update           7.3.13-1.el7.remi.x86_64                 @remi-php73
>     Updated php-mysqlnd-7.3.12-1.el7.remi.x86_64              @remi-php73
>     Update              7.3.13-1.el7.remi.x86_64              @remi-php73
>     Updated php-odbc-7.3.12-1.el7.remi.x86_64                 @remi-php73
>     Update           7.3.13-1.el7.remi.x86_64                 @remi-php73
>     Updated php-pdo-7.3.12-1.el7.remi.x86_64                  @remi-php73
>     Update          7.3.13-1.el7.remi.x86_64                  @remi-php73
>  ** Updated php-pear-1:1.10.10-2.el7.remi.noarch              @remi-php70
>  ** Update           1:1.10.10-4.el7.remi.noarch              @?remi-php73
>     Updated php-pgsql-7.3.12-1.el7.remi.x86_64                @remi-php73
>     Update            7.3.13-1.el7.remi.x86_64                @remi-php73
>     Updated php-process-7.3.12-1.el7.remi.x86_64              @remi-php73
>     Update              7.3.13-1.el7.remi.x86_64              @remi-php73
>     Updated php-xml-7.3.12-1.el7.remi.x86_64                  @remi-php73
>     Update          7.3.13-1.el7.remi.x86_64                  @remi-php73
>     Updated php-xmlrpc-7.3.12-1.el7.remi.x86_64               @remi-php73
>     Update             7.3.13-1.el7.remi.x86_64               @remi-php73
>     Updated postgresql10-10.11-1PGDG.rhel7.x86_64             @pgdg10
>     Update               10.11-2PGDG.rhel7.x86_64             @pgdg10
>     Updated postgresql10-contrib-10.11-1PGDG.rhel7.x86_64     @pgdg10
>     Update                       10.11-2PGDG.rhel7.x86_64     @pgdg10
>     Updated postgresql10-libs-10.11-1PGDG.rhel7.x86_64        @pgdg10
>     Update                    10.11-2PGDG.rhel7.x86_64        @pgdg10
>  ** Updated postgresql10-server-10.11-1PGDG.rhel7.x86_64      @pgdg10
>     Update                      10.11-2PGDG.rhel7.x86_64      @pgdg10
>  ** Updated proftpd-1.3.5e-7.el7.x86_64                       @epel
>     Update          1.3.5e-8.el7.x86_64                       @epel
>     Updated python-perf-3.10.0-1062.4.3.el7.x86_64            @updates
>     Update              3.10.0-1062.9.1.el7.x86_64            @updates
>     Updated python-requests-2.6.0-7.el7_7.noarch              @updates
>     Update                  2.6.0-8.el7_7.noarch              @updates
>     Updated python2-acme-0.39.0-1.el7.noarch                  @epel
>     Update               1.0.0-1.el7.noarch                   @epel
>     Updated python2-certbot-0.39.0-1.el7.noarch               @epel
>     Update                  1.0.0-1.el7.noarch                @epel
>     Updated python2-certbot-apache-0.39.0-1.el7.noarch        @epel
>     Update                         1.0.0-1.el7.noarch         @epel
>     Updated selinux-policy-3.13.1-252.el7.1.noarch            @updates
>     Update                 3.13.1-252.el7_7.6.noarch          @updates
>     Updated selinux-policy-targeted-3.13.1-252.el7.1.noarch   @updates
>     Update                          3.13.1-252.el7_7.6.noarch @updates
>     Updated sos-3.7-7.el7.centos.noarch                       @updates
>     Update      3.7-10.el7.centos.noarch                      @updates
>  ** Updated sssd-client-1.16.4-21.el7.x86_64                  @anaconda
>     Update              1.16.4-21.el7_7.1.x86_64              @updates
>     Updated tcpdump-14:4.9.2-4.el7.x86_64                     @anaconda
>     Update          14:4.9.2-4.el7_7.1.x86_64                 @updates
>     Updated util-linux-2.23.2-61.el7.x86_64                   @anaconda
>     Update             2.23.2-61.el7_7.1.x86_64               @updates
> Scriptlet output:
>    1 error: Couldn't fork %post(libsss_idmap-1.16.4-21.el7_7.1.x86_64):
> Cannot allocate memory
>    2 error: Couldn't fork
> %post(libsss_nss_idmap-1.16.4-21.el7_7.1.x86_64): Cannot allocate
> memory
>    3 error: Couldn't fork %pre(ca-certificates-2019.2.32-76.el7_7.noarch):
> Cannot allocate memory
>    4 error: ca-certificates-2019.2.32-76.el7_7.noarch: install failed
>    5 error: Couldn't fork %post(certbot-1.0.0-1.el7.noarch): Cannot
> allocate memory
>    6 error: Couldn't fork
> %pre(MariaDB-server-10.4.11-1.el7.centos.x86_64): Cannot allocate
> memory
>    7 error: MariaDB-server-10.4.11-1.el7.centos.x86_64: install failed
>    8 error: Couldn't fork %post(sssd-client-1.16.4-21.el7_7.1.x86_64):
> Cannot allocate memory
>    9 error: Couldn't fork %post(kernel-tools-3.10.0-1062.9.1.el7.x86_64):
> Cannot allocate memory
>   10 error: Couldn't fork %pre(php-pear-1:1.10.10-4.el7.remi.noarch):
> Cannot allocate memory
>   11 error: php-pear-1:1.10.10-4.el7.remi.noarch: install failed
>   12 warning: /etc/clamd.d/scan.conf created as
> /etc/clamd.d/scan.conf.rpmnew
>   13 error: Couldn't fork
> %triggerin(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate
> memory
>   14 error: php-pear-1:1.10.10-2.el7.remi.noarch: erase skipped
>   15 error: MariaDB-server-10.4.10-1.el7.centos.x86_64: erase skipped
>   16 error: Couldn't fork
> %preun(MariaDB-shared-10.4.10-1.el7.centos.x86_64): Cannot allocate
> memory
>   17 error: MariaDB-shared-10.4.10-1.el7.centos.x86_64: erase failed
>   18 error: Couldn't fork
> %postun(MariaDB-compat-10.4.10-1.el7.centos.x86_64): Cannot allocate
> memory
>   19 error: ca-certificates-2018.2.22-70.0.el7_5.noarch: erase skipped
>   20 error: Couldn't fork %preun(microcode_ctl-2:2.1-53.3.el7_7.x86_64):
> Cannot allocate memory
>   21 error: microcode_ctl-2:2.1-53.3.el7_7.x86_64: erase failed
>   22 error: Couldn't fork
> %preun(postgresql10-server-10.11-1PGDG.rhel7.x86_64): Cannot allocate
> memory
>   23 error: postgresql10-server-10.11-1PGDG.rhel7.x86_64: erase failed
>   24 error: Couldn't fork %postun(libmount-2.23.2-61.el7.x86_64): Cannot
> allocate memory
>   25 error: Couldn't fork %preun(sssd-client-1.16.4-21.el7.x86_64): Cannot
> allocate memory
>   26 error: sssd-client-1.16.4-21.el7.x86_64: erase failed
>   27 error: Couldn't fork %postun(libblkid-2.23.2-61.el7.x86_64): Cannot
> allocate memory
>   28 error: Couldn't fork %postun(postgresql10-10.11-1PGDG.rhel7.x86_64):
> Cannot allocate memory
>   29 error: Couldn't fork %postun(clamav-lib-0.101.4-1.el7.x86_64): Cannot
> allocate memory
>   30 error: Couldn't fork
> %postun(kernel-tools-3.10.0-1062.4.3.el7.x86_64): Cannot allocate memory
>   31 error: Couldn't fork %postun(libcurl-7.29.0-54.el7.x86_64): Cannot
> allocate memory
>   32 error: Couldn't fork %postun(nss-3.44.0-4.el7.x86_64): Cannot
> allocate memory
>   33 error: Couldn't fork %postun(nss-softokn-3.44.0-5.el7.x86_64): Cannot
> allocate memory
>   34 error: Couldn't fork %postun(nss-util-3.44.0-3.el7.x86_64): Cannot
> allocate memory
>   35 error: Couldn't fork
> %postun(postgresql10-libs-10.11-1PGDG.rhel7.x86_64): Cannot allocate
> memory
>   36 error: Couldn't fork %postun(libuuid-2.23.2-61.el7.x86_64): Cannot
> allocate memory
>   37 error: Couldn't fork %postun(libsss_idmap-1.16.4-21.el7.x86_64):
> Cannot allocate memory
>   38 error: Couldn't fork %postun(libsss_nss_idmap-1.16.4-21.el7.x86_64):
> Cannot allocate memory
>   39 error: Couldn't fork %postun(libsmartcols-2.23.2-61.el7.x86_64):
> Cannot allocate memory
>   40 error: Couldn't fork %postun(libwebp7-1.0.2-1.el7.remi.x86_64):
> Cannot allocate memory
>   41 error: Couldn't fork %postun(fribidi-1.0.2-1.el7.x86_64): Cannot
> allocate memory
>   42 error: Couldn't fork %preun(proftpd-1.3.5e-7.el7.x86_64): Cannot
> allocate memory
>   43 error: proftpd-1.3.5e-7.el7.x86_64: erase failed
>   44 error: Couldn't fork %postun(numactl-libs-2.0.12-3.el7.x86_64):
> Cannot allocate memory
>   45 error: Couldn't fork %posttrans(kernel-3.10.0-1062.9.1.el7.x86_64):
> Cannot allocate memory
>   46 error: Couldn't fork
> %posttrans(microcode_ctl-2:2.1-53.7.el7_7.x86_64): Cannot allocate
> memory
> history info
> [root at vps2 ~]#
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>