[CentOS] IBM BladeCenter HS20 blades

Fri Apr 15 22:14:06 UTC 2005
John Burk <jburk at mainframe.ca>

This problem also exhibits in RHEL3-U4, so CentOS 3.4 inherited it.

CentOS 3.4 is treated as rhas3 by xcat.  The first time you run 'nodeset 
<host> install' for a particular OS, the /tftpboot/xcat/rhas3/<arch> 
tree gets populated with files, including install.gz, which is a 
compressed disk image which in turn contains all the kernel modules, 
including tg3.

1.) First thing you need to do is delete (or rename) the 
/tftpboot/xcat/rhas3/<arch>/ directory.

2.) Next is to fix a minor bug in xcat's 'mkinstall' located in 
$XCATROOT/install/rhas3/ARCH/bin/.  I posted the fix to the xcat user's 
list, and have included the hacked mkinstall as an attachment.  Please 
note that this mkinstall is for 'x86', but it should be pretty 
straightforward to see how to make the change for x86_64 if that's your 

<<  cut&paste'd from my post in the xcat user's list  >>
I went the second route and dug around inside 
$XCATROOT/install/rhas3/ARCH/bin/mkinstall.  Turns out that there is a 
fix already inside this script and a module already built, but that it 
only will be applied to RHEL3-U4 and not CentOS 3.4

It's because of the file test at line 375 of mkinstall: the version 
string in the kernel rpm that ships on the RHEL3-U4 cd's is 2.4.21-27, 
but the CentOS-3.4 kernel version string is 2.4.21-27.0.1.  This version 
mismatch is echo'd at lines 383 and 386 also.
<< eo snip >>

3.) 'cd' to the $XCATROOT/install/rhas3/ARCH/ directory.  The fix only 
gets applied if you're in this directory, as the path the to bcm5700.o 
is relative - Egan Ford probably knows why he did it this way, I'm just 
hacking on his script and wanted to muck with it as little as possible.

4.) run 'nodeset <some_CentOS3.4_host> install'.  Because there is not 
an install.gz in the /tftpboot/xcat/rhas3/<arch> directory, it will be 
built, and because you're in the $XCATROOT/install/rhas3/ARCH/ 
directory, the pre-built bcm5700.o module which lives in 
$XCATROOT/install/rhas3/ARCH/data/27.EL will get put into the install.gz 
as tg3.o.

5.) 'rreset' your blades and watch them properly install CentOS 3.4 via 

John Burk

Sr. Technical Director
Mainframe Entertainment


Niswonger Charles M Contr AFRL/MNGG wrote:

> John,
>         I'm running 14 older HS20's right now and have 14 of the new 
> 8843 3.6 GHz EM64T blades coming in.  Right now we're running SuSE 9.2 
> and xcat, but we've seen the same tg3 bug that you mentioned.  I'm 
> curious as to the workaround that you used for CentOS.  With the 
> multi-level load needed to install the bootable image onto the blade 
> we still don't have a satisfactory workaround...
> Mike Niswonger, Systems Engineer
> charles.niswonger at eglin.af.mil
> Air Force Research Lab / MNGG (KHILS),
> SYColeman Corp.
> 101 W Eglin Blvd, Bldg. 13, Rm. 115
> Eglin AFB, FL 32542-5427
> (850)882-4446, x1217
> (850)882-4451, fax
> -----Original Message-----
> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On 
> Behalf Of John Burk
> Sent: Friday, April 15, 2005 1:42 PM
> To: CentOS mailing list
> Subject: Re: [CentOS] IBM BladeCenter HS20 blades
> I have 60 HS20's running CentOS 3.4, but I do the install via the xcat 
> toolset over nfs.
> Have you looked at xcat yet to manage the os on your blades?  Well 
> worth the time to setup, written primarily by IBM, and free.
> http://www.xcat.org/
> BTW, what processors are in your blades?  I'm running the 3.2GHz 
> Nocona's, if your are similar you'll probably need to use these kernel 
> parameters in your grub.conf:
>     acpi=noirq noexec=off noapic
> And if you do go with nfs installs via xcat, the tg3 driver in 3.4 is 
> broken.  E-mail me and I'll send you the work-around.  It's already in 
> xcat but it takes a bit of finesse to apply the fix.
> John Burk
> Sr. Technical Director
> Mainframe Entertainment
> 604.628.1019
> Brendon Colby wrote:
> > Greetings,
> >
> > We have purchased an IBM BladeCenter and I am in the process of
> > testing Linux installation on these things (boot off SAN i.e. qla2300
> > driver, not using internal drives). My distro of choice is Debian,
> > however, since I'm really not interested in trying to hand compile all
> > the drivers, I decided to try CentOS (which I'm so far very impressed
> > with).
> >
> > On boot, as with the Debian Sarge disk, the CDROM disappears. The
> > installation media complains that it cannot find the CD. I'm testing
> > using the DVD ISO of CentOS 4. Has anyone been successful in getting
> > CentOS 3/4 installed on a BladeCenter server? I noticed back in 11/04
> > there were a few posts on it.
> >
> > Thanks.
> >
> > Brendon Colby
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
> >
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mkinstall
URL: <http://lists.centos.org/pipermail/centos/attachments/20050415/ed16eb84/attachment-0004.ksh>