[CentOS] Problem with dual-booting soft-RAID

Sun Jun 4 23:29:25 UTC 2006
Kai Schaetzl <maillists at conactive.com>

I set up IDE software-RAID on a test machine for testing the recover 
functionality. The problem is that it somehow won't boot after removing 
the first disk, although grub etc. seems to be setup okay. Here's some 
background:

2 disks hda and hdb
RAID 1 setup:
/dev/md0 (hda1 + hdb1) /boot
/dev/md1 (hda2 + hdb2) /
/dev/md2 (hda3 + hdb3) swap

done with disk druid during initial setup from server disk, so far so 
good. Then I installed grub to hdb. grub-install complained all the time 
(I may have given it the wrong parameter), so I used grub directly:
root (hd1,0)
setup (hd1)
That's correct, isn't it? (Is there a way to determine if grub is 
installed on the correct disk?)
I added a third boot entry to the grub.conf (see below).

I then removed disk 1 (hda) from the array by unpowering it and rebooted. 
Unfortunately, that sent me right in the BIOS PXE boot instead of trying 
the other disk, well, that's a BIOS problem. So I repowered disk 1 and 
rebooted. The grub on it is still working somehow but just fills the 
screen with "grub". So I moved disk 2 before disk 1 as a boot device in 
BIOS and tried again. This time I get a grub prompt. So, grub is obviously 
correctly installed on it. However, why doesn't it then boot?

When I type in the exact same data that I have in grub.conf it boots. I 
resynced the disks now, but still can't boot normally.

grub.conf is like this (taken from another non-RAID, full setup grub.conf 
and adjusted, so it differs cosmetically):

default=0
fallback=2
timeout=10
#splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.9-34.0.1.EL)
    root (hd0,0)
    kernel /vmlinuz-2.6.9-34.0.1.EL ro root=/dev/md1
    initrd /initrd-2.6.9-34.0.1.EL.img
title CentOS-4 Server_CD (2.6.9-34.EL)
    root (hd0,0)
    kernel /vmlinuz-2.6.9-34.EL ro root=/dev/md1
    initrd /initrd-2.6.9-34.EL.img
title CentOS 2 (2.6.9-34.0.1.EL)
    root (hd1,0)
    kernel /vmlinuz-2.6.9-34.0.1.EL ro root=/dev/md1
    initrd /initrd-2.6.9-34.0.1.EL.img

When I change default to 2 it still doesn't boot, just gets me grub.
I then reinstalled grub on the resynced hda with the same commands used 
earlier (adjusted for it, of course) and rebooted with hda as first boot 
disk. There's still the same behavior as before: booting from hda gives 
"grub grub grub ...".

So, none of two boots like it should. disk 1 cannot boot at all, although 
resynced and disk 2 only manually.
What could be the problem?

Kai