We have purchased a product with our SAN to take a snapshot of one LUN so we can image that to another LUN for super fast installs. Therefore, I wanted to avoid having to set up xCat.
The problem with booting the CentOS 4 DVD was that the CDROM was not being detected. Since the CDROM in the media tray is USB, I switched over to another blade, and back, and the install then found the CDROM, and the rest went perfectly.
Thanks for your help on this.
Brendon Colby
John Burk wrote:
This problem also exhibits in RHEL3-U4, so CentOS 3.4 inherited it.
CentOS 3.4 is treated as rhas3 by xcat. The first time you run 'nodeset <host> install' for a particular OS, the /tftpboot/xcat/rhas3/<arch> tree gets populated with files, including install.gz, which is a compressed disk image which in turn contains all the kernel modules, including tg3.
1.) First thing you need to do is delete (or rename) the /tftpboot/xcat/rhas3/<arch>/ directory.
2.) Next is to fix a minor bug in xcat's 'mkinstall' located in $XCATROOT/install/rhas3/ARCH/bin/. I posted the fix to the xcat user's list, and have included the hacked mkinstall as an attachment. Please note that this mkinstall is for 'x86', but it should be pretty straightforward to see how to make the change for x86_64 if that's your arch.
<< cut&paste'd from my post in the xcat user's list >> I went the second route and dug around inside $XCATROOT/install/rhas3/ARCH/bin/mkinstall. Turns out that there is a fix already inside this script and a module already built, but that it only will be applied to RHEL3-U4 and not CentOS 3.4
It's because of the file test at line 375 of mkinstall: the version string in the kernel rpm that ships on the RHEL3-U4 cd's is 2.4.21-27, but the CentOS-3.4 kernel version string is 2.4.21-27.0.1. This version mismatch is echo'd at lines 383 and 386 also. << eo snip >>
3.) 'cd' to the $XCATROOT/install/rhas3/ARCH/ directory. The fix only gets applied if you're in this directory, as the path the to bcm5700.o is relative - Egan Ford probably knows why he did it this way, I'm just hacking on his script and wanted to muck with it as little as possible.
4.) run 'nodeset <some_CentOS3.4_host> install'. Because there is not an install.gz in the /tftpboot/xcat/rhas3/<arch> directory, it will be built, and because you're in the $XCATROOT/install/rhas3/ARCH/ directory, the pre-built bcm5700.o module which lives in $XCATROOT/install/rhas3/ARCH/data/27.EL will get put into the install.gz as tg3.o.
5.) 'rreset' your blades and watch them properly install CentOS 3.4 via nfs...
John Burk
Sr. Technical Director Mainframe Entertainment
604.628.1019
Niswonger Charles M Contr AFRL/MNGG wrote:
John,
I'm running 14 older HS20's right now and have 14 of the new8843 3.6 GHz EM64T blades coming in. Right now we're running SuSE 9.2 and xcat, but we've seen the same tg3 bug that you mentioned. I'm curious as to the workaround that you used for CentOS. With the multi-level load needed to install the bootable image onto the blade we still don't have a satisfactory workaround...
Mike Niswonger, Systems Engineer charles.niswonger@eglin.af.mil Air Force Research Lab / MNGG (KHILS), SYColeman Corp. 101 W Eglin Blvd, Bldg. 13, Rm. 115 Eglin AFB, FL 32542-5427 (850)882-4446, x1217 (850)882-4451, fax
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of John Burk Sent: Friday, April 15, 2005 1:42 PM To: CentOS mailing list Subject: Re: [CentOS] IBM BladeCenter HS20 blades
I have 60 HS20's running CentOS 3.4, but I do the install via the xcat toolset over nfs.
Have you looked at xcat yet to manage the os on your blades? Well worth the time to setup, written primarily by IBM, and free.
BTW, what processors are in your blades? I'm running the 3.2GHz Nocona's, if your are similar you'll probably need to use these kernel parameters in your grub.conf:
acpi=noirq noexec=off noapicAnd if you do go with nfs installs via xcat, the tg3 driver in 3.4 is broken. E-mail me and I'll send you the work-around. It's already in xcat but it takes a bit of finesse to apply the fix.
John Burk
Sr. Technical Director Mainframe Entertainment
604.628.1019
Brendon Colby wrote:
Greetings,
We have purchased an IBM BladeCenter and I am in the process of testing Linux installation on these things (boot off SAN i.e. qla2300 driver, not using internal drives). My distro of choice is Debian, however, since I'm really not interested in trying to hand compile all the drivers, I decided to try CentOS (which I'm so far very impressed with).
On boot, as with the Debian Sarge disk, the CDROM disappears. The installation media complains that it cannot find the CD. I'm testing using the DVD ISO of CentOS 4. Has anyone been successful in getting CentOS 3/4 installed on a BladeCenter server? I noticed back in 11/04 there were a few posts on it.
Thanks.
Brendon Colby _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
#!/bin/ksh #egan@us.ibm.com #See $XCATROOT/README for license. #
if [ -z "$XCATROOT" ] then if [ -r /etc/sysconfig/xcat ] then . /etc/sysconfig/xcat else if [ -r /etc/rc.config ] then . /etc/rc.config fi fi fi
if [ -z "$XCATROOT" ] then if [ -r "$(dirname $0)/../lib/xcatroot" ] then . $(dirname $0)/../lib/xcatroot fi fi
if [ -z "$XCATROOT" ] then echo "$(basename $0): env XCATROOT not defined!" >&2 exit 1 fi
if [ ! -d "$XCATROOT" ] then echo "$(basename $0): XCATROOT $XCATROOT does not exist!" >&2 exit 1 fi
. $XCATROOT/lib/functions
function usage { echo "\nUsage: ${MYNAME} [singlenode]\n" }
needhelp $* && usage >&2 && exit 0
if [ "$#" != "1" ] then usage >&2 exit 1 fi
NODE=$1 NODENAME=$1
if ! isanode $1 2>/dev/null then echo "${MYNAME}: $1 is not a node" >&2 exit 1 fi
if NODETYPE=$(tabdb $NODETYPETAB $NODE 3) then : else echo "${MYNAME}: $NODE nodetype image not defined in $NODETYPETAB" >&2 exit 1 fi
if OSVER=$(tabdb $NODETYPETAB $NODE 1) then : else echo "${MYNAME}: $NODE nodetype osver not defined in $NODETYPETAB" >&2 exit 1 fi
if ARCH=$(tabdb $NODETYPETAB $NODE 2) then : else echo "${MYNAME}: $NODE nodetype arch not defined in $NODETYPETAB" >&2 exit 1 fi
if NODERES=$(whatismyres2 $NODE) then RESNAME=$NODERES else echo "${MYNAME}: $NODE noderes not defined in $NODERESTAB" >&2 exit 1 fi
MYOSVER=rhas3 MYARCH=x86
if [ "$OSVER" != "$MYOSVER" -o "$ARCH" != "$MYARCH" ] then echo "${MYNAME}: Install directory does not match $NODE in $NODETYPETAB" exit 1 fi
if INSTALLDIR=$(tabdb $SITETAB installdir 1) then : else echo "$MYNAME: installdir not defined in $SITETAB" exit 1 fi
DISTDIR=$OSVER/$ARCH CFDIR=scripts/$OSVER/$ARCH PXET=install.pxe GRUBT=install.grub ELILOT=elilo.conf KERNEL=installz INITRD=install.gz
FILES="$INSTALLDIR/$DISTDIR/images/pxeboot/initrd.img $INSTALLDIR/$DISTDIR/images/pxeboot/vmlinuz /usr/share/terminfo/x/xterm $TFTPDIR/$TFTPXCATROOT $TFTPDIR"
if [ -r $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$KERNEL -a -r $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD ] then DOIMAGE=0 else DOIMAGE=1 fi
EXIT=0 for i in $FILES do if [ ! -r $i ] then echo "${MYNAME}: $i missing" EXIT=1 fi if [ "$EXIT" = "1" ] then exit 1 fi done
if [ -r $XCATPREFIX/tftp/pxelinux.0 ] then mkdir $TFTPDIR >/dev/null 2>&1 cp -f $XCATPREFIX/tftp/pxelinux.0 $TFTPDIR else echo "$MYNAME: $XCATPREFIX/tftp/pxelinux.0 missing" exit 1 fi
if [ -r $XCATPREFIX/tftp/default ] then mkdir $TFTPDIR/pxelinux.cfg >/dev/null 2>&1 cp -f $XCATPREFIX/tftp/default $TFTPDIR/pxelinux.cfg/default else echo "$MYNAME: $XCATPREFIX/tftp/default missing" exit 1 fi
MASTER=$(tabdb $SITETAB master 1) \ || echo "${MYNAME}: master not defined in $SITETAB" MASTER_IP=$(nametoip $MASTER)
#MASTER_IPS=$( # IPS="" # for i in $(dnsnr "@$MASTER(-eth[0-9])?") # do # if IP=$(nametoip $i) # then # IPS="$IPS $IP" # fi # done # print $IPS #)
MASTER_IPS=$( IPS="" for i in $(ifconfig | egrep '^eth[0-9]' | awk '{print $1}') do IP=$(ifconfig $i | perl -pi -e 's/ +/:/g' | grep inet:addr: | awk -F: '{print $4}') IPS="$IPS $IP" done print $IPS )
if CONFIG_SERVER=$(tabdb $NODERESTAB $RESNAME $noderes_config_server) then CONFIG_IP=$(nametoip $CONFIG_SERVER) else CONFIG_IP=$MASTER_IP CONFIG_SERVER=$MASTER fi
RSHC=$(tabdb $SITETAB rsh 1) \ || echo "${MYNAME}: rsh not defined in $SITETAB"
if [ "$(basename $RSHC)" = "rsh" ] then RSH="Y" else RSH="N" fi
SERIAL=$(tabdb $NODERESTAB $RESNAME $noderes_serial) \ || echo "${MYNAME}: serial defined in $NODERESTAB"
INSTALL_NFS=$(tabdb $NODERESTAB $RESNAME $noderes_nfs_install) \ || echo "${MYNAME}: nfs not defined in $NODERESTAB" INSTALL_NFS_IP=$(nametoip $INSTALL_NFS)
if [ "$INSTALL_NFS_IP" = "noip" ] then echo "Invalid Install NFS for $NODENAME" fi
INSTALL_NIC=$(tabdb $NODERESTAB $RESNAME $noderes_netdevice) \ || echo "${MYNAME}: install_nic not defined in $NODERESTAB"
INSTALL_DIR=$(tabdb $NODERESTAB $RESNAME $noderes_install_dir) \ || echo "${MYNAME}: nfs dir not defined in $NODERESTAB"
INSTALL_SRC_DIR=$INSTALL_DIR/$DISTDIR INSTALL_CF_DIR=$INSTALL_DIR/$CFDIR
mkdir -p $INSTALL_CF_DIR
FILE=$INSTALLDIR/$CFDIR/$NODETYPE-$NODERES INSTALL_CF_FILE=$NODETYPE-$NODERES
if [ ! -r $NODETYPE.tmpl ] then if [ ! -r base/$NODETYPE.tmpl ] then echo "${MYNAME}: cannot read template $NODETYPE.tmpl" exit 1 else TMPLDIR="base" fi else TMPLDIR="." fi
export MASTER export MASTER_IP export MASTER_IPS export CONFIG_IP CONFIG_SERVER export RSH export OSVER export ARCH export NODE export NODERES export NODENAME export NODETYPE export INSTALL_NFS export INSTALL_NFS_IP export INSTALL_SRC_DIR export INSTALL_CF_DIR export INSTALL_CF_FILE export KERNEL export INITRD
perl -pi -e 's/\\n$//' <$TMPLDIR/$NODETYPE.tmpl | LANG=C subvars.pl >$FILE
mkdir -p $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH
if [ -r "$PXET" ] then PXEF=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.pxe perl -pi -e 's/\\n$//' <$PXET | LANG=C subvars.pl >$PXEF if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ] then : else perl -pi -e "s/^SERIAL/#SERIAL/g" $PXEF # perl -pi -e "s/(console[^ ]* )//" $PXEF perl -pi -e "s/(console[^ ]* ){2}//" $PXEF fi fi
if [ -r "$GRUBT" ] then GRUBF=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.grub perl -pi -e 's/\\n$//' <$GRUBT | LANG=C subvars.pl >$GRUBF if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ] then : else perl -pi -e "s/^serial/#serial/g" $GRUBF perl -pi -e "s/^terminal/#terminal/g" $GRUBF # perl -pi -e "s/(console[^ ]* )//" $GRUBF perl -pi -e "s/(console[^ ]* ){2}//" $GRUBF fi fi
if [ -r "$ELILOT" ] then ELILOC=$TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$NODETYPE-$NODERES.elilo perl -pi -e 's/\\n$//' <$ELILOT | LANG=C subvars.pl >$ELILOC if [ "$SERIAL" = "0" -o "$SERIAL" = "1" ] then : else perl -pi -e "s/^SERIAL/#SERIAL/g" $ELILOC # perl -pi -e "s/console[^ ]* //" $ELILOC perl -pi -e "s/(console[^ ]* ){2}//" $ELILOC fi fi
if [ "$DOIMAGE" = "0" ] then exit 0 fi
cp -f $INSTALLDIR/$DISTDIR/images/pxeboot/vmlinuz $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$KERNEL cp -f $INSTALLDIR/$DISTDIR/images/pxeboot/initrd.img $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD
TMPDIR=/tmp/.mkinstall-$OSVER-$ARCH
if getlock mkinstall-$OSVER-$ARCH 60 then : else echo "${MYNAME} Failed to get $TMPDIR lock" exit 1 fi
rm -rf $TMPDIR mkdir -p $TMPDIR
gunzip -c $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD >$TMPDIR/initrd 2>/dev/null mkdir -p $TMPDIR/root mount -o loop $TMPDIR/initrd $TMPDIR/root #mkdir -p $TMPDIR/root/etc/terminfo/x #mkdir -p $TMPDIR/root/etc/terminfo/v #mkdir -p $TMPDIR/root/etc/terminfo/b #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/v/vt100 #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/b/bterm #cp -f /usr/share/terminfo/x/xterm $TMPDIR/root/etc/terminfo/x/xterm
### #rm -f $TMPDIR/root/etc/terminfo/v/vt100-nav
perl -pi -e 's/^.*qla2.00.*\n$//' $TMPDIR/root/modules/pcitable
#MYDIR=$(pwd) #cd $TMPDIR
#gunzip -c root/modules/modules.cgz | cpio -id 2>/dev/null
#if [ -r $MYDIR/data/e100.o ] #then # rm -f 2.4.21-3.ELBOOT/eepro100.o # cp -f $MYDIR/data/e100.o 2.4.21-3.ELBOOT/eepro100.o # cat $MYDIR/data/e100.pcitable >> root/modules/pcitable #fi
#if [ -r $MYDIR/data/bcm5700.o ] #then # cp -f $MYDIR/data/bcm5700.o 2.4.21-3.ELBOOT/bcm5700.o # cat $MYDIR/data/bcm5700.pcitable >> root/modules/pcitable # cat $MYDIR/data/bcm5700.module-info >> root/modules/module-info #fi
#if [ -r $MYDIR/data/e1000.o ] #then # cp -f $MYDIR/data/e1000.o 2.4.21-3.ELBOOT/e1000.o # cat $MYDIR/data/e1000.pcitable >> root/modules/pcitable # cat $MYDIR/data/e1000.module-info >> root/modules/module-info #fi
#find 2.4.21-3.ELBOOT -type f -print | cpio -o -H crc 2>/dev/null | gzip -c -9 >root/modules/modules.cgz
#cd $MYDIR
# default kernel version that ships with RHEL3-U4 KERNELVER=2.4.21-27 # # try and detect if we're actually installing CentOS instead of RHEL # ( tests cut&paste'd from XCATROOT/sbin/copycds ) # if [ -f $INSTALLDIR/$DISTDIR/.discinfo ] then if egrep "1104951135.38" $INSTALLDIR/$DISTDIR/.discinfo >/dev/null 2>&1 then if grep "i386" $INSTALLDIR/$DISTDIR/.discinfo >/dev/null 2>&1 then # kernel version in the CentOS-3.4 distro KERNELVER=2.4.21-27.0.1 fi
fi fi
# # fix the broken tg3 driver in RHEL3-U4/CentOS-3.4 # if [ -f $INSTALLDIR/$DISTDIR/RedHat/RPMS/kernel-${KERNELVER}.EL.i686.rpm ] then MYDIR=$(pwd) cd $TMPDIR gunzip -c root/modules/modules.cgz | cpio -id 2>/dev/null
if [ -r $MYDIR/data/27.EL/bcm5700.o ] then cp -f $MYDIR/data/27.EL/bcm5700.o ${KERNELVER}.ELBOOT/i386/tg3.o fi
find ${KERNELVER}.ELBOOT -type f -print | cpio -o -H crc 2>/dev/null | gzip -c -9 >root/modules/modules.cgz
cd $MYDIR fi
umount $TMPDIR/root gzip -9 -f $TMPDIR/initrd cp -f $TMPDIR/initrd.gz $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH/$INITRD
rm -rf $TMPDIR
freelock mkinstall-$OSVER-$ARCH 0
cd $TFTPDIR/$TFTPXCATROOT/$OSVER/$ARCH
chown -f root.root *.gz *.pxe *.nbi *.eb chmod -f 444 *.gz *.pxe *.nbi *.eb
exit 0
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos