I've launch a "yum update" while I was going to lunch. 1 hour later the system was completely freezed (no mouse, no keyboard, screen black, no network).
I try a hard reboot => kernel panic on the new kernel
I boot on the old kernel => boot hangs with gnome-shell failure. SSH was OK so I remove the new kernel and re-install gnome shell.
Reboot is ok but now the computer freeze (no mouse, no keyboard, screen black) after some time. Yum asks to run yum-complete-transaction
Running yum-complete-transaction says:
There are 1 outstanding transactions to complete. Finishing the most recent one The remaining transaction had 1537 elements left to run
then many package to be removed and start a very long dependances check and show a lot of errors at the end with a depsolving loop limit reached:
Erreur : Depsolving loop limit reached. Erreur : Paquet : gtk3-3.22.30-3.el7.i686 (base) Requiert : libXext.so.6 Suppression : libXext-1.3.3-3.el7.i686 (@base) libXext.so.6 Erreur : Paquet : evolution-mapi-3.28.3-2.el7.x86_64 (base) Requiert : libemail-engine.so()(64bit) Suppression : evolution-3.22.6-14.el7.x86_64 (@base) libemail-engine.so()(64bit) Suppression : evolution-3.28.5-2.el7.x86_64 (installed) libemail-engine.so()(64bit)
.......
and then:
Vous pouvez essayer d'utiliser --skip-broken pour contourner le problème ** 775 problèmes RPMDB préexistants trouvés, la sortie de « yum check » est la suivante : GeoIP-1.5.0-13.el7.x86_64 est un doublon de GeoIP-1.5.0-11.el7.x86_64 1:NetworkManager-1.12.0-8.el7_6.x86_64 est un doublon de 1:NetworkManager-1.10.2-16.el7_5.x86_64 1:NetworkManager-glib-1.12.0-8.el7_6.x86_64 des conflits sont installés NetworkManager-libnm < ('1', '1.12.0', '8.el7_6'): 1:NetworkManager-libnm-1.10.2-16.el7_5.x86_64 1:NetworkManager-glib-1.12.0-8.el7_6.x86_64 est un doublon de 1:NetworkManager-glib-1.10.2-16.el7_5.x86_64 1:NetworkManager-libnm-1.12.0-8.el7_6.x86_64 des conflits sont installés NetworkManager-glib < ('1', '1.12.0', '8.el7_6'):
...etc and fails.
Any idea to go back to a normal situation before a full re-install ?
Thanks
Patrick
On 08.12.18 13:48, Patrick Bégou wrote:
There are 1 outstanding transactions to complete. Finishing the most
Any idea to go back to a normal situation before a full re-install ?
restoring the old state before the broken 'yum update' can be done by package-cleanup.
package-cleanup --dupes
will give you a long list of duplicate packages.
package-cleanup --cleandupes --removenewestdupes
will try to remove the duplicate rpms from the update.
after that you should run 'package-cleanup --dupes' and clean the remaining dupes manually. if this is done, you can try to run the 'yum update' again.
HTH and best regards Ulf
Hi Ulf
thanks for the suggestion. Indeed it seems I have a problem with glibc. Version is different between x86_64 and i386 and I have two glibc-common for x86_64. Trying to remove the old one request removing half of the system packages because of the dependancies. The newest cannot be removed.
glibc-2.17-222.el7.i686 glibc-2.17-260.el7.x86_64 glibc-common-2.17-222.el7.x86_64 glibc-common-2.17-260.el7.x86_64 glibc-devel-2.17-260.el7.x86_64 glibc-headers-2.17-260.el7.x86_64
Patrick
De: "Ulf Volmer" u.volmer@u-v.de À: centos@centos.org Envoyé: Samedi 8 Décembre 2018 14:07:11 Objet: Re: [CentOS] Centos7 broken after update
On 08.12.18 13:48, Patrick Bégou wrote:
There are 1 outstanding transactions to complete. Finishing the most
Any idea to go back to a normal situation before a full re-install ?
restoring the old state before the broken 'yum update' can be done by package-cleanup.
package-cleanup --dupes
will give you a long list of duplicate packages.
package-cleanup --cleandupes --removenewestdupes
will try to remove the duplicate rpms from the update.
after that you should run 'package-cleanup --dupes' and clean the remaining dupes manually. if this is done, you can try to run the 'yum update' again.
HTH and best regards Ulf _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 08.12.18 15:18, PATRICK BÉGOU wrote:
thanks for the suggestion. Indeed it seems I have a problem with glibc. Version is different between x86_64 and i386 and I have two glibc-common for x86_64. Trying to remove the old one request removing half of the system packages because of the dependancies. The newest cannot be removed.
glibc-2.17-222.el7.i686 glibc-2.17-260.el7.x86_64 glibc-common-2.17-222.el7.x86_64 glibc-common-2.17-260.el7.x86_64 glibc-devel-2.17-260.el7.x86_64 glibc-headers-2.17-260.el7.x86_64
my first try would be to remove glibc-common-2.17-222.el7.x86_64. and try to run yum update then again.
(BTW: please do not top post).
best regards Ulf
thanks for the suggestion. Indeed it seems I have a problem with glibc. Version is different between x86_64 and i386 and I have two glibc-common for x86_64. Trying to remove the old one request removing half of the system packages because of the dependancies. The newest cannot be removed.
glibc-2.17-222.el7.i686 glibc-2.17-260.el7.x86_64 glibc-common-2.17-222.el7.x86_64 glibc-common-2.17-260.el7.x86_64 glibc-devel-2.17-260.el7.x86_64 glibc-headers-2.17-260.el7.x86_64
Yes, I battled with this exact same problem earlier in the week - machine crashed halfway through the big update leaving things in a very inconsistent state (lots of dupes, some library files zero length - which was fun). Took me about 12 hours in total to sort out - I was that close to just re-installing.
For glibc problem, can you do
yum erase glibc-2.17-222.el7.i686
there shouldn't be any serious dependencies on the .i686 version. If there are, just make a note and reinstall them later.
"package-cleanup --cleandupes" helped sort out some of the issues and with the remaining few it was a case of manually removing the specific versions and reinstalling what dependencies there were.
Once most of the issues were done, I then did a 'yum distrosync" (or whatever it's called) to make sure packages were correct.
I still had problems with zero length files in some places, so I installed yum-verify and ran 'yum verify' to show packages that were inconsistent - there were lots. To get a list of them I did
yum verify > yum.verify.out grep ' : ' yum.verify.out
The 'yum verify' takes a looong time, so be patient.
P.
Le 08/12/2018 à 16:23, Pete Biggs a écrit :
thanks for the suggestion. Indeed it seems I have a problem with glibc. Version is different between x86_64 and i386 and I have two glibc-common for x86_64. Trying to remove the old one request removing half of the system packages because of the dependancies. The newest cannot be removed.
glibc-2.17-222.el7.i686 glibc-2.17-260.el7.x86_64 glibc-common-2.17-222.el7.x86_64 glibc-common-2.17-260.el7.x86_64 glibc-devel-2.17-260.el7.x86_64 glibc-headers-2.17-260.el7.x86_64
Yes, I battled with this exact same problem earlier in the week - machine crashed halfway through the big update leaving things in a very inconsistent state (lots of dupes, some library files zero length - which was fun). Took me about 12 hours in total to sort out - I was that close to just re-installing.
For glibc problem, can you do
yum erase glibc-2.17-222.el7.i686
there shouldn't be any serious dependencies on the .i686 version. If there are, just make a note and reinstall them later.
"package-cleanup --cleandupes" helped sort out some of the issues and with the remaining few it was a case of manually removing the specific versions and reinstalling what dependencies there were.
Once most of the issues were done, I then did a 'yum distrosync" (or whatever it's called) to make sure packages were correct.
I still had problems with zero length files in some places, so I installed yum-verify and ran 'yum verify' to show packages that were inconsistent - there were lots. To get a list of them I did
yum verify > yum.verify.out grep ' : ' yum.verify.out
The 'yum verify' takes a looong time, so be patient.
P.
Thanks all for these advices. I've cleaned all these duplicates. It removed several hundreds of rpm. Two duplicates were not removables because they were depedencies of protected package (ie :yum...). After checking with "yum verify" I reinstalled near 50 packages manualy. Then "yum update" was successfull and I reinstalled (with "yum groupinstall") most of the things. Then ran again a duplicate removal with success (there were 2 packages remaining in duplicate state). The laptop is stable now and works even if my environment is a little bit different than previously. Difficult to know what is missing now.
Monday I'll update the backup of my datas (last backup was 2 days ago) and re-install this laptop (it's fast, just a pxe boot and kickstart) at work as I need to have the same config than the other users. Many thanks for your help wich allow me to understand how to solve such a situation. Crashing this laptop was not critical, just unable to work this week-end, but if this happens later on one of my servers, I have now a validated strategy to apply. This was why solving the problem was important!
And sorry Ulf for previous top posting.
Patrick
Le 08/12/2018 à 16:23, Pete Biggs a écrit :
thanks for the suggestion. Indeed it seems I have a problem with glibc. Version is different between x86_64 and i386 and I have two glibc-common for x86_64. Trying to remove the old one request removing half of the system packages because of the dependancies. The newest cannot be removed.
glibc-2.17-222.el7.i686 glibc-2.17-260.el7.x86_64 glibc-common-2.17-222.el7.x86_64 glibc-common-2.17-260.el7.x86_64 glibc-devel-2.17-260.el7.x86_64 glibc-headers-2.17-260.el7.x86_64
Yes, I battled with this exact same problem earlier in the week - machine crashed halfway through the big update leaving things in a very inconsistent state (lots of dupes, some library files zero length - which was fun). Took me about 12 hours in total to sort out - I was that close to just re-installing.
For glibc problem, can you do
yum erase glibc-2.17-222.el7.i686
there shouldn't be any serious dependencies on the .i686 version. If there are, just make a note and reinstall them later.
"package-cleanup --cleandupes" helped sort out some of the issues and with the remaining few it was a case of manually removing the specific versions and reinstalling what dependencies there were.
Once most of the issues were done, I then did a 'yum distrosync" (or whatever it's called) to make sure packages were correct.
I still had problems with zero length files in some places, so I installed yum-verify and ran 'yum verify' to show packages that were inconsistent - there were lots. To get a list of them I did
yum verify > yum.verify.out grep ' : ' yum.verify.out
The 'yum verify' takes a looong time, so be patient.
P.
Thanks all for these advices. I've cleaned all these duplicates. It removed several hundreds of rpm. Two duplicates were not removables because they were depedencies of protected package (ie :yum...). After checking with "yum verify" I reinstalled near 50 packages manualy. Then "yum update" was successfull and I reinstalled (with "yum groupinstall") most of the things. Then ran again a duplicate removal with success (there were 2 packages remaining in duplicate state). The laptop is stable now and works even if my environment is a little bit different than previously. Difficult to know what is missing now.
Monday I'll update the backup of my datas (last backup was 2 days ago) and re-install this laptop (it's fast, just a pxe boot and kickstart) at work as I need to have the same config than the other users. Many thanks for your help wich allow me to understand how to solve such a situation. Crashing this laptop was not critical, just unable to work this week-end, but if this happens later on one of my servers, I have now a validated strategy to apply. This was why solving the problem was important!
I'm still wondering *what* is responsible for the ugly situation where the system freezes while updating?
The only idea I have is that systemd has a problem somewhere, and that's it for the whole system. I have a system which acts as a desktop and application server for ~50 users. While running for some time there are a lot of the following processes lying around left by user session:
/usr/bin/python /usr/share/system-config-printer/scp-dbus-service.py
Once I had the glorious idea to clean those left over processes up. That was quite a bad idea, because it brought systemd into a state which didn't allow me to do anything with the system anymore. In the end only solution was to shutdown the whole system. Dbus and systemd were somewhow messed up and there was no way to do any systemctl call anymore. I never felt so stupid and limited running Unix and Linux systems for decades. That said, I'm not a systemd guru as I prefer Unix/Linux as operating systems :)
Now if something like that happens during distro upgrade, I expect exactly the kind of fun you describe above. Of course I could be completely wrong, it's just a feeling I got.
In fact, the system I mentioned above is the last one I didn't upgrade to 7.6 and I'm still a bit afraid doing so.
Regards, Simon
-
Patrick Bégou -
Pete Biggs -
Simon Matter -
Ulf Volmer