Evan Bursey <ehbursey at lbl.gov> wrote: > I've just started caring for a dual Opteron machine. It > has a 3Ware 9500S-12 RAID controller, a Tyan S2882G3NR > motherboard, 10 IBM 400GB disk drives, and a 650 watt power > supply. Ouch, that's pushing it bad. I use a 550W with an old dual-P3 (sub-20W/each) and 3Ware Escalade 7800 with 8 x PATA at home. Most of my other installations that have 8-12 drives are typically 6U or pedestal with 850W or 1000W 2+1 2xSSI power supplies. > (actually, I did get some joy out of recompiling the kernel. > Its been several years since I last compiled a kernel and > the process has changed for the better). Actually, you shouldn't have to rebuild the kernel. You just: 1. Run make in the 3Ware source directory (the Makefile 3Ware includes does the job) 2. Copy the new 3w-9xxx.ko module in the appropriate /lib/modules/x.x.x-x directory 3. Run "depmod -a" to rebuild dependencies 4. Run "mkinitrd /boot/x.x.x-x-new.img x.x.x-x" to build a new initrd 5. Edit /etc/grub.conf to add a new entry with the new initrd > I've been in contact with 3Ware. From the error logs I > supplied to them, it looks like the issue is an insufficient > power supply. Would not surprise me one bit. Although the 3Ware can stage spin-ups at boot, if you have enough power usage, the drives can still stall due to lack of power in full seek. > Their rule of thumb is 300W for the motherboard You'll easily need 400+W with dual-Opterons. I'm sure your voltage is dropping on the +12V due to inadequate current. > and 35 W for each disk drive. Yep, sounds typical at 3A at 12V ~ 36W. > That means that I would need (at a theoretical minimum) a > 650 W power supply - which I have. I'd say you're way over budget. > I only notice the problem in SMP mode, not in uniprocessor > mode. Correct because even though the CPU is plugged in, it's probably idle and not sucking up any juice. > That suggests to me that (1) I have a > boarderline-sufficient power supply that can support one > but not two CPUs I concur, it's definitely power. Unless you're staging your drive startup, I'm surprised you don't have an issue at boot. > or (2) there is a problem with the combination of > Opteron/3Ware 9500S-12 hardware and the x86_64 SMP > kernel. I suppose there are even worse possibilities, > but I'd rather not think about those right now. > What do you think? Definitely power. You are way over budget. You should either consider a 2+1 PS design (3 units, 2 required to run), or use a separate power supply for the drives. What kind of enclosure do you have currently? > Does anyone have any experience with this sort of hardware > on x86_64 kernels? I don't think it's a kernel issue. I think the x86-64 kernel is making far greater use of your Opteron's capabilities. I mean, there are literally units that get "turned on" when the x86-64 kernel is running -- things the i686/Athlon kernel does not enable at all. > Does the 3w_9xxx driver work well on a x86_64 machine? > Does anyone have a suggestion as to what capacity power > supply I should have? Yes, 850W, easily. Power PC & Cooling has a non-redundant SSI one for just over $400. I know there are a few other vendors that have one for about $300. I'd be interested in what your enclosure is right now. -- Bryan J. Smith | Sent from Yahoo Mail mailto:b.j.smith at ieee.org | (please excuse any http://thebs413.blogspot.com/ | missing headers)