Hi all, A few months ago, I migrated some of our internal servers to HP blades, as the VMWare box they were previously running on was getting too slow.
However, it wasn't without it's problems, and eventually the only way I could get them to work was:
Install the same version of CentOS on the blade (believed to be 5.0, but /etc/redhat-release says 5.2) Took down both servers, booting them off the SystemRescueCD, mounting all the partition on /mnt/transfer etc Ran rsync to copy all the data from one to the other - except /boot and /lib/modules Brought the blade up, and saw that it booted fine.
However, a new kernel has been released and we tried rebooting... And it panicked. It seems to be LVM related, in that it can't mount /dev/root - and I've tried manually running mkinitrd to regenerate the initrd to no avail.
I was just wondering if anyone had encountered similar problems, and knew of any solutions?
I'm from a debian background, so am a little unused to certain aspects - when a yum upgrade installs a new kernel, where does it pull the information for boot from? A configuration stanza from within grub's menu.lst?
Regards,
Matthew Walster
On Wed, Nov 19, 2008 at 7:59 AM, Matthew Walster matthew@walster.org wrote:
A few months ago, I migrated some of our internal servers to HP blades, as the VMWare box they were previously running on was getting too slow. However, it wasn't without it's problems, and eventually the only way I could get them to work was: Install the same version of CentOS on the blade (believed to be 5.0, but /etc/redhat-release says 5.2) Took down both servers, booting them off the SystemRescueCD, mounting all the partition on /mnt/transfer etc Ran rsync to copy all the data from one to the other - except /boot and /lib/modules Brought the blade up, and saw that it booted fine. However, a new kernel has been released and we tried rebooting... And it panicked. It seems to be LVM related, in that it can't mount /dev/root - and I've tried manually running mkinitrd to regenerate the initrd to no avail. I was just wondering if anyone had encountered similar problems, and knew of any solutions? I'm from a debian background, so am a little unused to certain aspects - when a yum upgrade installs a new kernel, where does it pull the information for boot from? A configuration stanza from within grub's menu.lst?
I think you will find that in /boot/grub/grub.conf
Other, far more knowledgeable people on the list will, hopefully, answer your questions.
2008/11/19 Lanny Marcus lmmailinglists@gmail.com
I think you will find that in /boot/grub/grub.conf
$ mount /dev/cciss/c0d0p2 on / type ext3 (rw,noatime)
$ sudo cat /etc/grub/menu.list #symlinked to /etc/grub/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/cciss/c0d0p2 # initrd /initrd-version.img #boot=/dev/cciss/c0d0 default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-92.1.17.el5) root (hd0,0) kernel /vmlinuz-2.6.18-92.1.17.el5 ro root=/dev/cciss/c0d0p2 initrd /initrd-2.6.18-92.1.17.el5.img title CentOS (2.6.18-92.1.13.el5) root (hd0,0) kernel /vmlinuz-2.6.18-92.1.13.el5 ro root=/dev/cciss/c0d0p2 initrd /initrd-2.6.18-92.1.13.el5.img title CentOS (2.6.18-53.el5) root (hd0,0) kernel /vmlinuz-2.6.18-53.el5 ro root=LABEL=/1 initrd /initrd-2.6.18-53.el5.img
$ uname -a Linux (HOSTNAME) 2.6.18-53.el5 #1 SMP Mon Nov 12 02:14:55 EST 2007 x86_64 x86_64 x86_64 GNU/Linux
And yes, I've tried altering the root= lines to match the working kernel, but that didn't work either, and it gets overwritten anyway after every kernel upgrade.
Other, far more knowledgeable people on the list will, hopefully,
answer your questions.
Thanks ;)
Matthew Walster
2008/11/19 Matthew Walster matthew@walster.org
$ sudo cat /etc/grub/menu.list #symlinked to /etc/grub/grub.conf
I, of course, meant /boot/grub/menu.lst and /boot/grub/grub.conf
Matthew Walster
Matthew Walster wrote on Wed, 19 Nov 2008 15:39:18 +0000:
And yes, I've tried altering the root= lines to match the working kernel, but that didn't work either, and it gets overwritten anyway after every kernel upgrade.
change default to 1 if that was your working kernel. If that doesn't help the problem is not what/where you think it is.
Kai
On Wed, Nov 19, 2008 at 10:39 AM, Matthew Walster matthew@walster.org wrote:
2008/11/19 Lanny Marcus lmmailinglists@gmail.com
I think you will find that in /boot/grub/grub.conf
$ mount /dev/cciss/c0d0p2 on / type ext3 (rw,noatime) $ sudo cat /etc/grub/menu.list #symlinked to /etc/grub/grub.conf # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/cciss/c0d0p2 # initrd /initrd-version.img #boot=/dev/cciss/c0d0 default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-92.1.17.el5) root (hd0,0) kernel /vmlinuz-2.6.18-92.1.17.el5 ro root=/dev/cciss/c0d0p2 initrd /initrd-2.6.18-92.1.17.el5.img title CentOS (2.6.18-92.1.13.el5) root (hd0,0) kernel /vmlinuz-2.6.18-92.1.13.el5 ro root=/dev/cciss/c0d0p2 initrd /initrd-2.6.18-92.1.13.el5.img title CentOS (2.6.18-53.el5) root (hd0,0) kernel /vmlinuz-2.6.18-53.el5 ro root=LABEL=/1 initrd /initrd-2.6.18-53.el5.img $ uname -a Linux (HOSTNAME) 2.6.18-53.el5 #1 SMP Mon Nov 12 02:14:55 EST 2007 x86_64 x86_64 x86_64 GNU/Linux And yes, I've tried altering the root= lines to match the working kernel, but that didn't work either, and it gets overwritten anyway after every kernel upgrade.
Other, far more knowledgeable people on the list will, hopefully, answer your questions.
Thanks ;)
Matthew: You're welcome. As Kai wrote earlier, you need to change the line that says default=n to the "title" line for the kernel you want to use (I think it defaults to 0, after a Kernel upgrade). Every time you install a new kernel, it will change this file, so you may need to go back and tweak it, after a kernel upgrade. Lanny
Hi all,
A few months ago, I migrated some of our internal servers to HP blades, as the VMWare box they were previously running on was getting too slow.
However, it wasn't without it's problems, and eventually the only way I could get them to work was:
Install the same version of CentOS on the blade (believed to be 5.0, but /etc/redhat-release says 5.2) Took down both servers, booting them off the SystemRescueCD, mounting all the partition on /mnt/transfer etc Ran rsync to copy all the data from one to the other - except /boot and /lib/modules Brought the blade up, and saw that it booted fine.
However, a new kernel has been released and we tried rebooting... And it panicked. It seems to be LVM related, in that it can't mount /dev/root - and I've tried manually running mkinitrd to regenerate the initrd to no avail.
I was just wondering if anyone had encountered similar problems, and knew of any solutions?
Having just done the opposite (test migrating a CentOS server from physical hardware to VMWare), I have a very good idea :-) It was CentOS 4, but the basic bits should be the same
You need to edit /etc/modprobe.conf, and change the line (or lines) that start with: "alias scsi_hostadapter"
If the server was vmware, the existing lines are probably alias scsi_hostadapter mptbase alias scsi_hostadapter1 mptscsih
I'm not sure what you'll have to change mptbase and mptscsih to be; on HP servers, cciss is the correct driver. Others will vary (you could try a quick basic manual install on the target hardware and see what gets put in that file by the OS installer).
After you've changed modprobe.conf, re-run mkinitrd and reboot. Should be good to go. You'll also have to watch out for HWADDR in /etc/sysconfig/network-scripts/ifcfg-eth*, either changing it to match your new hardware, or removing it entirely. Kudzu will probably take care of sorting out the network card driver entries in modprobe; follow your nose and take a guess, you'll probably get it right. Either way, mkinitrd is only necessary (in this case) to get access to your root device
Hope that helps, Craig Miskell
======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================
2008/11/19 Miskell, Craig Craig.Miskell@agresearch.co.nz
I was just wondering if anyone had encountered similar problems, and knew of any solutions?
Having just done the opposite (test migrating a CentOS server from physical hardware to VMWare), I have a very good idea :-) It was CentOS 4, but the basic bits should be the same
Thanks for that, I'll give it a go. It's been bugging me for some time.
When I get around to testing this (preferably when it's not a peak hour!) I'll come back and let you guys know if it worked, know if there's a place I can document it? It can't just be us two that have had this issue.
Matthew Walster
Matthew Walster wrote on Thu, 20 Nov 2008 11:44:36 +0000:
It can't just be us two that have had this issue.
I don't have this issue. It was just standard advice on how you go back to an older kernel.
And yes, I've tried altering the root= lines to match the working kernel,
and I gave it after reading that line. It's easy to mistype or do something else wrong to the file when editing that much. Just changing the default to boot that supposed to be working kernel gives much less source for mistakes. If it then still doesn't boot it's more likely that it's got nothing to do with the kernel version.
Kai
Matthew Walster wrote: ...
When I get around to testing this (preferably when it's not a peak hour!) I'll come back and let you guys know if it worked, know if there's a place I can document it? It can't just be us two that have had this issue.
You can reply to the list with SOLVED added to the subject line for future searchers of the list archives. This is probably not general enough to warrant a Wiki article, although there have been a lot of recent questions about hardware/kernel changes causing hangups and requiring tweaks to grub, modules.conf, and needing to mkinitrd - if someone feels up to writing a more general guide to solving such problems.
Phil
-
Kai Schaetzl -
Lanny Marcus -
Matthew Walster -
Miskell, Craig -
Phil Schaffner