Hi all,
Is anyone successfully running/has succesfully upgraded to 2.6.32-220 from, say, 2.6.32-71.29.1? (i.e. done a normal run-of-the-mill yum update on, say a 6.0 instance all the way up cleanly to 6.2?
Reason I ask is that booting into -220 (and I think also into -131 as well) results in a kernel panic for me. Some digging around and the new kernel seems to be enumerating the drives with the wrong minors
An m1.large instance-store instance has root at /dev/xvda1, and 2 ephemeral drives xvdb and xvdc. On reboot into 2.6.32-220.7.1 this is what the kernel reports this:
dracut: dracut-004-256.el6 device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.22.6-ioctl (2011-10-19) initialised: dm-devel@redhat.com udev: starting version 147 dracut: Starting plymouth daemon %Gxlblk_init: register_blkdev major: 202 blkfront: xvde1: barriers disabled blkfront: xvdf: barriers disabled xvdf: unknown partition table %Gblkfront: xvdg: barriers disabled xvdg: unknown partition table dracut Warning: No root device "block:/dev/xvda1" found
%G%G
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line. Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tainted 2.6.32-220.7.1.el6.x86_64 #1 Call Trace: [<ffffffff814ec3fa>] ? panic+0x78/0x143 [<ffffffff810074db>] ? __raw_callee_save_xen_irq_enable+0x11/0x26 [<ffffffff8106ed72>] ? do_exit+0x852/0x860 [<ffffffff81177f75>] ? fput+0x25/0x30 [<ffffffff8106edd8>] ? do_group_exit+0x58/0xd0 [<ffffffff8106ee67>] ? sys_exit_group+0x17/0x20 [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
Altering the grub menu.1st and fstab and the instance will boot from /dev/xvde1 but obviously the device change is fairly fundamental.
The image that this instance is booting was built in a 6.0 chroot and I have about 100 of them successfully running; but this innocuous update breaks things in a major way.
I've had a poke around the upstream bugzilla (e.g. https://bugzilla.redhat.com/show_bug.cgi?id=771912) as well as the EC2 forums and there are a couple of similar but not-quite-the-same problems with the newer kernel.
Does anyone have any similar experience or advice?
Cheers,
Steph
Hello Steph,
On Mon, 16 Apr 2012 10:44:24 +0100 Steph Gosling steph@chuci.org wrote:
Hi all,
Is anyone successfully running/has succesfully upgraded to 2.6.32-220 from, say, 2.6.32-71.29.1? (i.e. done a normal run-of-the-mill yum update on, say a 6.0 instance all the way up cleanly to 6.2?
Reason I ask is that booting into -220 (and I think also into -131 as well) results in a kernel panic for me. Some digging around and the new kernel seems to be enumerating the drives with the wrong minors
An m1.large instance-store instance has root at /dev/xvda1, and 2 ephemeral drives xvdb and xvdc. On reboot into 2.6.32-220.7.1 this is what the kernel reports this:
dracut: dracut-004-256.el6 device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.22.6-ioctl (2011-10-19) initialised: dm-devel@redhat.com udev: starting version 147 dracut: Starting plymouth daemon %Gxlblk_init: register_blkdev major: 202 blkfront: xvde1: barriers disabled blkfront: xvdf: barriers disabled xvdf: unknown partition table %Gblkfront: xvdg: barriers disabled xvdg: unknown partition table dracut Warning: No root device "block:/dev/xvda1" found
%G%G
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line. Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tainted 2.6.32-220.7.1.el6.x86_64 #1 Call Trace: [<ffffffff814ec3fa>] ? panic+0x78/0x143 [<ffffffff810074db>] ? __raw_callee_save_xen_irq_enable+0x11/0x26 [<ffffffff8106ed72>] ? do_exit+0x852/0x860 [<ffffffff81177f75>] ? fput+0x25/0x30 [<ffffffff8106edd8>] ? do_group_exit+0x58/0xd0 [<ffffffff8106ee67>] ? sys_exit_group+0x17/0x20 [<ffffffff8100b0f2>] ? system_call_fastpath+0x16/0x1b
Altering the grub menu.1st and fstab and the instance will boot from /dev/xvde1 but obviously the device change is fairly fundamental.
The image that this instance is booting was built in a 6.0 chroot and I have about 100 of them successfully running; but this innocuous update breaks things in a major way.
I've had a poke around the upstream bugzilla (e.g. https://bugzilla.redhat.com/show_bug.cgi?id=771912) as well as the EC2 forums and there are a couple of similar but not-quite-the-same problems with the newer kernel.
Does anyone have any similar experience or advice?
Check if the thread "Recent kernel update vs usb disk" is related to your issue (I presume so), thread is from early March 2012.
Regards,
Hi,
On Mon, 16 Apr 2012 11:55:17 +0200 wwp subscript@free.fr wrote:
Hello Steph,
Check if the thread "Recent kernel update vs usb disk" is related to your issue (I presume so), thread is from early March 2012.
Regards,
Think the problems are different as this isn't related to the USB subsystem. I'll have a read though.
Cheers,
Steph
On 04/16/2012 10:44 AM, Steph Gosling wrote:
Does anyone have any similar experience or advice?
because the devices are now mapped as sda/sdb instead of xvda/xvdb ?
Hi Karanbir,
That's the thing, older (non pv-grub aware kernels) did used to map them with the old scsi device names, but here now it's still mapping them as 'xvdN' but N in this case is 'e', not 'a', 'f', not 'b' and so on.
Upstream seem to have a handful of bugs related to dracut and initramfs creation but I don't think that's the case here but I'll look at it again.
Cheers,
Steph
On Tue, 17 Apr 2012 00:00:48 +0100 Karanbir Singh mail-lists@karan.org wrote:
On 04/16/2012 10:44 AM, Steph Gosling wrote:
Does anyone have any similar experience or advice?
because the devices are now mapped as sda/sdb instead of xvda/xvdb ?
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
hi,
Please dont toppost, trim your reply and keep context in your replies.
On 04/17/2012 08:04 AM, Steph Gosling wrote:
them as 'xvdN' but N in this case is 'e', not 'a', 'f', not 'b' and so on.
And labels dont help here ?
On Tue, 17 Apr 2012 12:35:07 +0100 Karanbir Singh mail-lists@karan.org wrote:
hi,
Please dont toppost, trim your reply and keep context in your replies.
Apologies (mail sent before coffee this morning!)
On 04/17/2012 08:04 AM, Steph Gosling wrote:
them as 'xvdN' but N in this case is 'e', not 'a', 'f', not 'b' and so on.
And labels dont help here ?
Labels do help for the root device but not for ephemeral devices in EC2:- with an EC2 instance you'll know the label or UUID of the root device but ephemeral devices are created at startup time. This makes getting mountpoints right hard if you're programmatically starting instances difficult.
There's also a 'mapping' between what the EC2 APIs return as the root device (helpfully still referred to via the old SCSI names), again which makes programmatic operations break: expecting something to be /dev/sda1 in EC2 that was /dev/xvda1 in the instance but magically now is /dev/xvde1.
Anyway as I say it's definitely a problem with upstream so we'll just wait and see. Just explaining this here so that the search engines get it and it'll hopefully help someone else with the same problem in the future.
Cheers,
Steph
(bad form replying to myself)
I've found the issue upstream:
https://bugzilla.redhat.com/show_bug.cgi?id=729586
Last comment there saying there are patches in an as yet unreleased kernel-2.6.32-229.el6. I've had a quick look at the SRPMS upstream and don't see that one yet so a related question: how quickly do they release these or make them available for testing?
Cheers,
Steph
On 04/17/2012 06:27 AM, Steph Gosling wrote:
(bad form replying to myself)
I've found the issue upstream:
https://bugzilla.redhat.com/show_bug.cgi?id=729586
Last comment there saying there are patches in an as yet unreleased kernel-2.6.32-229.el6. I've had a quick look at the SRPMS upstream and don't see that one yet so a related question: how quickly do they release these or make them available for testing?
They usually do not make them available for testing anymore ... and if they do, it is usually a link from the bugzilla page and likely for a limited time. (Thanks Oracle)
Since it is a 229 version, I would think it will be released at the next point release ... so for 6.3
-
Johnny Hughes -
Karanbir Singh -
Steph Gosling -
wwp