I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2. However, I get a kernel panic on boot. The first kernel panic I got included stuff about acpi, so I tried adding noacpi noapic to the kernel boot parameters, which at least changed the kernel panic message, now I get (transcribed from a photo I took, so please excuse any errors):
Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tainted 2.6.32-431.29.2.el6.x86_64 #1 Call Trace: (excluding addresses, please let me know if they're good for anything) panic do_exit fput do_group sys_exit_group system_call_fastpath
My grub.conf entry looks like this currently:
title CentOS (2.6.32-431.29.2.el6.x86_64) root (hd0,0) kernel /boot/vmlinuz-2.6.32-431.29.2.el6.x86_64 ro root=UUID=ca1e1248-0a65-4b6c-9f87-0c859eab1f17 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rhgb quiet nomodeset rdblacklist=nouveau KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM verbose selinux=0 enforcing=0 noacpi noapic nolapic nouveau.modeset=0 initrd /boot/initramfs-2.6.32-431.29.2.el6.x86_64.img
(With all the options I added to see if it would work, including a couple from a very similar box that runs fine with that kernel, but on CentOS 6.5.)
This box has a bunch of cards in it, including some NVidia GPUs in a PCI expander, another NVidia GPU for the GUI, an Areca controller, a Mellanox IB adapter, and some other stuff.
But, I'm pretty sure this must be something simple I'm missing. Ideas for figuring it out?
On Mon, Oct 13, 2014 at 10:18:54PM -0500, Joakim Ziegler wrote:
I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2.
Is that a 6.4 kernel? Seems like it ought to be 6.5 from the date.
But, I'm pretty sure this must be something simple I'm missing. Ideas for figuring it out?
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
On 10/14/2014 09:19 AM, Greg Lindahl wrote:
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
If yum/rpm allowed him to just upgrade the core kernel witouh the whole system, that means it should be possible to run with it. Please, be positive.
On Tue, Oct 14, 2014 at 09:26:41AM +0300, Mihamina Rakotomandimby wrote:
On 10/14/2014 09:19 AM, Greg Lindahl wrote:
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
If yum/rpm allowed him to just upgrade the core kernel witouh the whole system, that means it should be possible to run with it.
That is so not the case!
Please, be positive.
Uhuh. If you ask for advice, you will receive it.
-- greg
On 10/14/2014 02:29 AM, Greg Lindahl wrote:
On Tue, Oct 14, 2014 at 09:26:41AM +0300, Mihamina Rakotomandimby wrote:
On 10/14/2014 09:19 AM, Greg Lindahl wrote:
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
If yum/rpm allowed him to just upgrade the core kernel witouh the whole system, that means it should be possible to run with it.
That is so not the case!
I have been doing just that for years with CentOS 5 and CentOS 6 and never, ever had a problem on a whole bunch of different hardware!
Please, be positive.
Uhuh. If you ask for advice, you will receive it.
-- greg
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 10/13/2014 11:29 PM, Greg Lindahl wrote:
On Tue, Oct 14, 2014 at 09:26:41AM +0300, Mihamina Rakotomandimby wrote:
Please, be positive.
Uhuh. If you ask for advice, you will receive it.
-- greg
with respect -- that was Not advice.
On 14/10/14, 1:19, Greg Lindahl wrote:
On Mon, Oct 13, 2014 at 10:18:54PM -0500, Joakim Ziegler wrote:
I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2.
Is that a 6.4 kernel? Seems like it ought to be 6.5 from the date.
But, I'm pretty sure this must be something simple I'm missing. Ideas for figuring it out?
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
I don't know. yum info doesn't say anything about it being a 6.4 or 6.5 kernel (nor does it say that about the 2.6.32-358 package). How can I tell if a package is intended for 6.4 or 6.5? The release field simply contains "el6", nothing about minor version.
On 10/14/2014 10:12 AM, Joakim Ziegler wrote:
On 14/10/14, 1:19, Greg Lindahl wrote:
On Mon, Oct 13, 2014 at 10:18:54PM -0500, Joakim Ziegler wrote:
I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2.
Is that a 6.4 kernel? Seems like it ought to be 6.5 from the date.
But, I'm pretty sure this must be something simple I'm missing. Ideas for figuring it out?
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
I don't know. yum info doesn't say anything about it being a 6.4 or 6.5 kernel (nor does it say that about the 2.6.32-358 package). How can I tell if a package is intended for 6.4 or 6.5? The release field simply contains "el6", nothing about minor version.
that's because there is no spoon, there's just el6... http://wiki.centos.org/FAQ/General#head-dcca41e9a3d5ac4c6d900a991990fd119308...
On 14/10/14, 3:24, Nicolas Thierry-Mieg wrote:
On 10/14/2014 10:12 AM, Joakim Ziegler wrote:
On 14/10/14, 1:19, Greg Lindahl wrote:
On Mon, Oct 13, 2014 at 10:18:54PM -0500, Joakim Ziegler wrote:
I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2.
Is that a 6.4 kernel? Seems like it ought to be 6.5 from the date.
But, I'm pretty sure this must be something simple I'm missing. Ideas for figuring it out?
Yeah: don't run random combinations of rpms and then ask the mailing list for support.
I don't know. yum info doesn't say anything about it being a 6.4 or 6.5 kernel (nor does it say that about the 2.6.32-358 package). How can I tell if a package is intended for 6.4 or 6.5? The release field simply contains "el6", nothing about minor version.
that's because there is no spoon, there's just el6... http://wiki.centos.org/FAQ/General#head-dcca41e9a3d5ac4c6d900a991990fd119308...
Ok, so is that a confirmation that installing this kernel, even though it might be "for 6.5" should not in itself break anything, and that it should boot?
Joakim Ziegler joakim@terminalmx.com a écrit :
Ok, so is that a confirmation that installing this kernel, even though it might be "for 6.5" should not in itself break anything, and that it should boot?
Every RH errata contains the following text: « Before applying this update, make sure all previously released errata relevant to your system have been applied. »
And that’s the case for that kernel: https://rhn.redhat.com/errata/RHSA-2014-1167.html
So, imho, just yum update and reboot. You’ll be at 6.5 and far more safer.
My 0.02€. Laurent.
On 14/10/14, 3:32, Laurent Wandrebeck wrote:
Joakim Ziegler joakim@terminalmx.com a écrit :
Ok, so is that a confirmation that installing this kernel, even though it might be "for 6.5" should not in itself break anything, and that it should boot?
Every RH errata contains the following text: « Before applying this update, make sure all previously released errata relevant to your system have been applied. »
And that’s the case for that kernel: https://rhn.redhat.com/errata/RHSA-2014-1167.html
So, imho, just yum update and reboot. You’ll be at 6.5 and far more safer.
Yeah, I might end up doing that. However, this is a system that runs proprietary software that explicitly runs on 6.4, so I was trying to avoid upgrading everything.
Oh well. It won't be the first time I make this particular propritary system run on a distro it doesn't officially support, they were stuck on CentOS 5 for the longest time, and I made it work on 6.3 before they got with the times.
On 10/14/2014 03:38 AM, Joakim Ziegler wrote:
On 14/10/14, 3:32, Laurent Wandrebeck wrote:
Joakim Ziegler joakim@terminalmx.com a écrit :
Ok, so is that a confirmation that installing this kernel, even though it might be "for 6.5" should not in itself break anything, and that it should boot?
Every RH errata contains the following text: « Before applying this update, make sure all previously released errata relevant to your system have been applied. »
And that’s the case for that kernel: https://rhn.redhat.com/errata/RHSA-2014-1167.html
So, imho, just yum update and reboot. You’ll be at 6.5 and far more safer.
Yeah, I might end up doing that. However, this is a system that runs proprietary software that explicitly runs on 6.4, so I was trying to avoid upgrading everything.
Oh well. It won't be the first time I make this particular propritary system run on a distro it doesn't officially support, they were stuck on CentOS 5 for the longest time, and I made it work on 6.3 before they got with the times.
The advise to do a full upgrade is the best (most secure) option .. however, theoretically, the new kernel should boot and not cause issues based on the other packages.
I would search for kernel issues for your model server and the 6.5 kernels, as I think the issue is with some hardware drivers and not related to the other userland packages (unless you happen to be using a kmod somewhere).
That being said, you should always try to stay current with all updates to maximize the chances that everything works together .. and Laurent is absolutely correct, the only tested 'upstream' solution (other than their EUS?AUS solutions) is what he said, which is:
"Before applying this update, make sure all previously released errata relevant to your system have been applied."
As far as what kernel is designed for which release ... every point release (minor version) will have its own kernel branch associated with it, you can see this at http://vault.centos.org/
For example, 6.0 has:
kernel-2.6.32-71.el6
All the other 6.0 kernels are kernel-2.6.32-71.x.y.el6 .. ie, kernel-2.6.32-71.7.1.el6, kernel-2.6.32-71.14.1, kernel-2.6.32-71.18.1.el6, kernel-2.6.32-71.18.2, etc.
The following is a breakdown of what each starts with:
6.0: kernel-2.6.32-71 6.1: kernel-2.6.32-131 6.2: kernel-2.6.32-220 6.3: kernel-2.6.32-279 6.4: kernel-2.6.32-358 6.5: kernel-2.6.32-431 6.6: kernel-2.6.32-504
Thanks, Johnny Hughes
On Tue, Oct 14, 2014 at 06:56:23AM -0500, Johnny Hughes wrote:
The advise to do a full upgrade is the best (most secure) option .. however, theoretically, the new kernel should boot and not cause issues based on the other packages.
OP said he had an InfiniBand card. For a long time it was the case that every kernel point release needed the corresponding IB userspace libraries.
It's also worth noting that running with an unusual set of packages means you're running something that is not well-tested. Most enterprise users like to run with the herd, in the middle of the herd. Mooo. That basically means "yum -y update", nothing different. "Yum let me do it" and "this is wise" are two different things!
-- greg
On 15/10/14, 1:58, Greg Lindahl wrote:
On Tue, Oct 14, 2014 at 06:56:23AM -0500, Johnny Hughes wrote:
The advise to do a full upgrade is the best (most secure) option .. however, theoretically, the new kernel should boot and not cause issues based on the other packages.
OP said he had an InfiniBand card. For a long time it was the case that every kernel point release needed the corresponding IB userspace libraries.
It's also worth noting that running with an unusual set of packages means you're running something that is not well-tested. Most enterprise users like to run with the herd, in the middle of the herd. Mooo. That basically means "yum -y update", nothing different. "Yum let me do it" and "this is wise" are two different things!
I can confirm that a full upgrade to 6.5 fixed the boot problem. I do think it's maybe a bit too easy to make this mistake, though.
On 10/13/2014 11:18 PM, Joakim Ziegler wrote:
I'm on a Supermicro server, X9DA7 motherboard, Intel C602 chipset, 2x 2.4GHz Intel Xeon E5-2665 8-core CPU, 96GB RAM, and I'm running CentOS 6.4.
I just tried to use yum to upgrade the kernel from 2.6.32-358 to 2.6.32-431.29.2. However, I get a kernel panic on boot. The first kernel panic I got included stuff about acpi, so I tried adding noacpi noapic to the kernel boot parameters, which at least changed the kernel panic message, now I get (transcribed from a photo I took, so please excuse any errors):
...
First question: can you boot with the old kernel still (by default CentOS 6 leaves a few old kernels around; I want to say the default is 3, but it might be 5, I don't recall, and I don't have a straight default C6 install to check against right at the moment)?
Next question: did you also update the updated kernel-firmware package for the updated kernel?
The first thing I would do is downgrade the kernel and make sure the system is working there; you then will need to very carefully check all your hardware components together that the kernel update should be ok. You mention GPU's; which drivers are you using there? Iterate over all hardware.
Now, I'm going to sound like a broken record here. If you absolutely positively must stay at a point release for whatever reason (and there are valid reasons for this), then you don't need to be running CentOS; it is simply not supported. You either need to pay up for RHEL6 with EUS, or you need to install ScientificLinux 6 (built from the same sources that CentOS is built from, with a different rebranding); the SL team does support getting only critical updates but staying on a particular point release.
On Wed, 15 Oct 2014 09:22:04 -0400 Lamar Owen lowen@pari.edu wrote: ...
Now, I'm going to sound like a broken record here. If you absolutely positively must stay at a point release for whatever reason (and there are valid reasons for this), then you don't need to be running CentOS; it is simply not supported. You either need to pay up for RHEL6 with EUS,
To be clear EUS only provides critical updates (+some important) and for a limited time. You can expect ~1y extra (6.3 no longer even EUS supported and 6.4 ends in feb 2015).
/Peter
On 15/10/14, 8:22, Lamar Owen wrote:
First question: can you boot with the old kernel still (by default CentOS 6 leaves a few old kernels around; I want to say the default is 3, but it might be 5, I don't recall, and I don't have a straight default C6 install to check against right at the moment)?
Next question: did you also update the updated kernel-firmware package for the updated kernel?
The first thing I would do is downgrade the kernel and make sure the system is working there; you then will need to very carefully check all your hardware components together that the kernel update should be ok. You mention GPU's; which drivers are you using there? Iterate over all hardware.
Now, I'm going to sound like a broken record here. If you absolutely positively must stay at a point release for whatever reason (and there are valid reasons for this), then you don't need to be running CentOS; it is simply not supported. You either need to pay up for RHEL6 with EUS, or you need to install ScientificLinux 6 (built from the same sources that CentOS is built from, with a different rebranding); the SL team does support getting only critical updates but staying on a particular point release.
Upgrading to 6.5 did fix the problem, and did not (so far) seem to break my proprietary software.
For reference, I had already updated the kernel-firmware package (that happened automatically), and I could still boot the old kernel, which was how I got around to upgrading to 6.5.
More than anything, it's a little annoying that this is such an easy mistake to make, and so hard (it seems) to debug. But well, I won't be making it again.
-
Greg Lindahl -
Igal ＠ getRailo.org -
Joakim Ziegler -
Johnny Hughes -
Lamar Owen -
Laurent Wandrebeck -
Mihamina Rakotomandimby -
Nicolas Thierry-Mieg -
Peter Kjellström -
Steve Clark