[CentOS] Centos7 broken after update

Mon Dec 10 07:25:37 UTC 2018
Simon Matter <simon.matter at invoca.ch>

> Le 08/12/2018 à 16:23, Pete Biggs a écrit :
>>> thanks for the suggestion. Indeed it seems I have a problem with
>>> glibc. Version is different between x86_64 and i386 and I have two
>>> glibc-common for x86_64.
>>> Trying to remove the old one request removing half of the system
>>> packages because of the dependancies. The newest cannot be removed.
>>>
>>>
>>> glibc-2.17-222.el7.i686
>>> glibc-2.17-260.el7.x86_64
>>> glibc-common-2.17-222.el7.x86_64
>>> glibc-common-2.17-260.el7.x86_64
>>> glibc-devel-2.17-260.el7.x86_64
>>> glibc-headers-2.17-260.el7.x86_64
>>>
>> Yes, I battled with this exact same problem earlier in the week -
>> machine crashed halfway through the big update leaving things in a very
>> inconsistent state (lots of dupes, some library files zero length -
>> which was fun). Took me about 12 hours in total to sort out - I was
>> that close to just re-installing.
>>
>> For glibc problem, can you do
>>
>>    yum erase glibc-2.17-222.el7.i686
>>
>> there shouldn't be any serious dependencies on the .i686 version.  If
>> there are, just make a note and reinstall them later.
>>
>> "package-cleanup --cleandupes" helped sort out some of the issues and
>> with the remaining few it was a case of manually removing the specific
>> versions and reinstalling what dependencies there were.
>>
>> Once most of the issues were done, I then did a 'yum distrosync" (or
>> whatever it's called) to make sure packages were correct.
>>
>> I still had problems with zero length files in some places, so I
>> installed yum-verify and ran 'yum verify' to show packages that were
>> inconsistent - there were lots. To get a list of them I did
>>
>>   yum verify > yum.verify.out
>>   grep ' : ' yum.verify.out
>>
>> The 'yum verify' takes a looong time, so be patient.
>>
>> P.
>>
> Thanks all for these advices. I've cleaned all these duplicates. It
> removed several hundreds of rpm. Two duplicates were not removables
> because they were depedencies of protected package (ie :yum...). After
> checking with "yum verify" I reinstalled near 50 packages manualy. Then
> "yum update" was successfull and I reinstalled (with "yum groupinstall")
> most of the things. Then ran again a duplicate removal with success
> (there were 2 packages remaining in duplicate state).
> The laptop is stable now and works even if my environment is a little
> bit different than previously. Difficult to know what is missing now.
>
> Monday I'll update the backup of my datas (last backup was 2 days ago)
> and re-install this laptop (it's fast, just a pxe boot and kickstart) at
> work as I need to have the same config than the other users. Many thanks
> for your help wich allow me to understand how to solve such a situation.
> Crashing this laptop was not critical, just unable to work this
> week-end, but if this happens later on one of my servers, I have now a
> validated strategy to apply. This was why solving the problem was
> important!

I'm still wondering *what* is responsible for the ugly situation where the
system freezes while updating?

The only idea I have is that systemd has a problem somewhere, and that's
it for the whole system. I have a system which acts as a desktop and
application server for ~50 users. While running for some time there are a
lot of the following processes lying around left by user session:

/usr/bin/python /usr/share/system-config-printer/scp-dbus-service.py

Once I had the glorious idea to clean those left over processes up. That
was quite a bad idea, because it brought systemd into a state which didn't
allow me to do anything with the system anymore. In the end only solution
was to shutdown the whole system. Dbus and systemd were somewhow messed up
and there was no way to do any systemctl call anymore. I never felt so
stupid and limited running Unix and Linux systems for decades. That said,
I'm not a systemd guru as I prefer Unix/Linux as operating systems :)

Now if something like that happens during distro upgrade, I expect exactly
the kind of fun you describe above. Of course I could be completely wrong,
it's just a feeling I got.

In fact, the system I mentioned above is the last one I didn't upgrade to
7.6 and I'm still a bit afraid doing so.

Regards,
Simon