Following the advice on this mailing list, I rebuilt (from scratch) Centos 7.8. I added the following line into /etc/yum.conf before the first update:
exclude=grub2* shim* mokutil
The system worked fine. I was able to do general updates yum -y update and all was well.
HOWEVER, following Johnny Hughes' directions:
a) remove the 'exclude' line from yum.conf b) perform update: yum -y update c) reinstall kernel. Here the instructions were vague, so I typed in... yum reinstall kernel-3.10.0 deriving the "3.10.0" label by the output of rpm -qa | grep kernel
Yum got up to the point:
Running transaction Installing : kernel-3.10.0-1127.el7.x86_64 1/1
at which point the process appeared to hang. No further output happened for five minutes. I opened a different terminal and entered "shutdown -r now". The result is an unbootable system.
What did I do wrong? I must admit that there are multiple copies of advice on the mailing list, so perhaps I followed the wrong one?
------------------------------
So now, I tried the 'repair' mechanism, but couldn't make that work either. So, it was back to initial from-scratch install.
David
On Tue, Aug 04, 2020 at 10:54:50AM -0700, david wrote:
Yum got up to the point:
Running transaction Installing : kernel-3.10.0-1127.el7.x86_64 1/1
at which point the process appeared to hang. No further output happened for five minutes. I opened a different terminal and entered "shutdown -r now". The result is an unbootable system.
What did I do wrong? I must admit that there are multiple copies of advice on the mailing list, so perhaps I followed the wrong one?
Your system was most likely rebuilding the initrd, and you interrupted it leaving you with a broken initrd.
Try booting off a rescue disk and chroot into the install, and run:
dracut -f -v
to regenerate all the initrds.
Also, you don't need to reinstall the kernel but just do a 'yum update' to get everything up to the latest release. The shim package with the fix is the latest.
<snip>
Your system was most likely rebuilding the initrd, and you interrupted it leaving you with a broken initrd.
Try booting off a rescue disk and chroot into the install, and run:
dracut -f -v
to regenerate all the initrds.
Also, you don't need to reinstall the kernel but just do a 'yum update' to get everything up to the latest release. The shim package with the fix is the latest.
-- Jonathan Billings billings@negate.org ___
Jonathan et al:
On the assumption that all the updates should now "just work", I did a from-scratch reinstall of a minimal Centos 7 system starting from netinstall and added software that I've always added (perl, dhcp, mariadb, etc). I did not exclude shim, etc in "/etc/yum.conf". After all the updates, the system was NOT bootable.
I repeated the install and updates, but this time including the line in /etc/yum.conf exclude=grub2* shim* mokutil and ended up with a usable system.
This was not the result I was hoping for.
David
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait: 5 minutes
What on the console: nothing, just a dull gray color
Errors on update: none
---------------------
But when I blocked the update, it booted within a minute, and ran.
Am 05.08.20 um 01:09 schrieb david:
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait: 5 minutes
What on the console: nothing, just a dull gray color
Errors on update: none
But when I blocked the update, it booted within a minute, and ran.
Can you boot the system with all updates and secureboot=off? (Just to be sure; I imply that you use UEFI, right?)
-- Leon
At 04:18 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:09 schrieb david:
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait:Â 5 minutes What on the console:Â nothing, just a dull gray color Errors on update:Â none
But when I blocked the update, it booted within a minute, and ran.
Can you boot the system with all updates and secureboot=off? (Just to be sure; I imply that you use UEFI, right?)
-- Leon ____________
I'm not sure how to turn 'secure boot' off or if it exists. (MacMini5.2). I presume it uses UEFI, but not sure how to answer that.
Boot failure only occurs when the grub2/shim/mokutil updates are applied.
David
Am 05.08.20 um 01:27 schrieb david:
At 04:18 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:09 schrieb david:
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait:Â 5 minutes What on the console:Â nothing, just a dull gray color Errors on update:Â none
But when I blocked the update, it booted within a minute, and ran.
Can you boot the system with all updates and secureboot=off? (Just to be sure; I imply that you use UEFI, right?)
-- Leon ____________
I'm not sure how to turn 'secure boot' off or if it exists. (MacMini5.2). I presume it uses UEFI, but not sure how to answer that.
Oh, an apple device. AFAIK the openfirmware of such hardware have also a legacy mode. So first check if it uses the UEFI mode at all by checking if this directory exists (in the working/bootable system):
# ls -la /sys/firmware/efi
if so test the secure boot state with
# mokutil --sb-state
Boot failure only occurs when the grub2/shim/mokutil updates are applied.
-- Leon
At 05:01 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:27 schrieb david:
At 04:18 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:09 schrieb david:
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot?ÃÂ What text messages showed up on the console?ÃÂ Any reported errors when you ran the update or when you rebooted the computer?ÃÂ If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait:ÃÂ 5 minutes What on the console:ÃÂ nothing, just a dull gray color Errors on update:ÃÂ none
But when I blocked the update, it booted within a minute, and ran.
Can you boot the system with all updates and secureboot=off? (Just to be sure; I imply that you use UEFI, right?)
-- Leon ____________
I'm not sure how to turn 'secure boot' off or if it exists. (MacMini5.2). I presume it uses UEFI, but not sure how to answer that.
Oh, an apple device. AFAIK the openfirmware of such hardware have also a legacy mode. So first check if it uses the UEFI mode at all by checking if this directory exists (in the working/bootable system):
# ls -la /sys/firmware/efi
if so test the secure boot state with
# mokutil --sb-state
Boot failure only occurs when the grub2/shim/mokutil updates are applied.
[root@xxx -]ls -la /sys/firmware/efi total 0 drwxr-xr-x 5 root root 0 Aug 4 17:12 . drwxr-xr-x 7 root root 0 Aug 4 14:30 .. -r--r--r-- 1 root root 4096 Aug 4 17:12 config_table drwxr-xr-x 2 root root 0 Aug 4 14:30 efivars -r--r--r-- 1 root root 4096 Aug 4 17:12 fw_platform_size -r--r--r-- 1 root root 4096 Aug 4 17:12 fw_vendor -r--r--r-- 1 root root 4096 Aug 4 17:12 runtime drwxr-xr-x 10 root root 0 Aug 4 17:12 runtime-map -r-------- 1 root root 4096 Aug 4 14:31 systab drwxr-xr-x 23 root root 0 Aug 4 17:12 vars [root@xxx ~]# mokutil --sb-state This system doesn't support Secure Boot [root@xxx ~]#
Am 05.08.20 um 02:13 schrieb david:
At 05:01 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:27 schrieb david:
At 04:18 PM 8/4/2020, you wrote:
Am 05.08.20 um 01:09 schrieb david:
At 01:54 PM 8/4/2020, you wrote:
On Tue, 04 Aug 2020 13:44:05 -0700 david wrote:
> After all the updates, the system was NOT bootable.
How long did you wait for it to boot, and what did it do when it failed to boot? What text messages showed up on the console? Any reported errors when you ran the update or when you rebooted the computer? If so, what did the say?
I personally haven't had any issues updating any of my computers (using a mix of Centos 6, 7 and 8) but maybe they're all too old to for the issue to show up.
--
How long did I wait: 5 minutes What on the console: nothing, just a dull gray color Errors on update: none
But when I blocked the update, it booted within a minute, and ran.
Can you boot the system with all updates and secureboot=off? (Just to be sure; I imply that you use UEFI, right?)
-- Leon ____________
I'm not sure how to turn 'secure boot' off or if it exists. (MacMini5.2). I presume it uses UEFI, but not sure how to answer that.
Oh, an apple device. AFAIK the openfirmware of such hardware have also a legacy mode. So first check if it uses the UEFI mode at all by checking if this directory exists (in the working/bootable system):
# ls -la /sys/firmware/efi
if so test the secure boot state with
# mokutil --sb-state
Boot failure only occurs when the grub2/shim/mokutil updates are applied.
[root@xxx -]ls -la /sys/firmware/efi total 0 drwxr-xr-x 5 root root 0 Aug 4 17:12 . drwxr-xr-x 7 root root 0 Aug 4 14:30 .. -r--r--r-- 1 root root 4096 Aug 4 17:12 config_table drwxr-xr-x 2 root root 0 Aug 4 14:30 efivars -r--r--r-- 1 root root 4096 Aug 4 17:12 fw_platform_size -r--r--r-- 1 root root 4096 Aug 4 17:12 fw_vendor -r--r--r-- 1 root root 4096 Aug 4 17:12 runtime drwxr-xr-x 10 root root 0 Aug 4 17:12 runtime-map -r-------- 1 root root 4096 Aug 4 14:31 systab drwxr-xr-x 23 root root 0 Aug 4 17:12 vars [root@xxx ~]# mokutil --sb-state This system doesn't support Secure Boot [root@xxx ~]#
The boot hole security issue is related to secure boot. In your case I would assume a different problem (after seeing the above information). As others mentioned already apply some patience while updating. You said that you could change to a different terminal. Take a look into "top", if something like gz or xz is in place occupying your CPU then the initrd gets build ... just wait :-)
-- Leon
On 8/4/2020 11:20 AM, Jonathan Billings wrote:
Running transaction Installing : kernel-3.10.0-1127.el7.x86_64 1/1
at which point the process appeared to hang. No further output happened for five minutes. I opened a different terminal and entered "shutdown -r now". The result is an unbootable system.
What did I do wrong? I must admit that there are multiple copies of advice on the mailing list, so perhaps I followed the wrong one?
Your system was most likely rebuilding the initrd, and you interrupted it leaving you with a broken initrd.
Is there some way we could get the initrd rebuild to be more verbose, so that it doesn't appear to hang? It would be nice to get feedback that something is happening, especially on an older, slower system that takes a long time for this step.
I run into the same problem with the kernel-devel package, just because of the sheer number of files involved. I've learned to be patient and expect a kernel upgrade on my oldest system to take a very long time. (I need the -devel package to rebuild an ancient 3rd party driver no longer provided by RHEL.)
On 05/08/2020 10:40, Kenneth Porter wrote:
Is there some way we could get the initrd rebuild to be more verbose, so that it doesn't appear to hang? It would be nice to get feedback that something is happening, especially on an older, slower system that takes a long time for this step.
Not without hacking the underlying scripts that call dracut to increase the verbosity. You can run yum at a more verbose debug level, but it's not going to give you information on the state of individual rpm scriptlets. You could monitor the process in top where you should be able to see that it's still working. You can also monitor /var/log/messages where dracut activity is logged.
On 8/5/20 5:40 AM, Kenneth Porter wrote:
Is there some way we could get the initrd rebuild to be more verbose, so that it doesn't appear to hang? It would be nice to get feedback that something is happening, especially on an older, slower system that takes a long time for this step.
While the process CAN be made more verbose, it won't help any. RPM scriptlets are actually not supposed to generate output; both rpm and yum (and by extension dnf) buffer scriptlet output so that any messages that are issued will all be seen after the entire scriptlet has run. The same thing is true for say the ELrepo kmods, such as kmod-nvidia. That rpm needs to rebuild the initrd, and even issues a message saying something like 'this may take a while' or similar, but you won't get that message until the scriptlet doing the initrd rebuild has finished.
This is one area I think the Debian packaging system is a bit superior, in that dpkg's can issue progress reports and similar. But rpm has been this way for a really really long time, ever since I maintained the PostgreSQL RPMset from 1999 to 2004.
-
david -
Frank Cox -
Jonathan Billings -
Kai Bojens -
Kenneth Porter -
Lamar Owen -
Leon Fauster -
Phil Perry