Sean O'Connell <oconnell at soe.ucsd.edu> wrote: > The machine has 2 Opteron 250DP (2.4GHz) with 4 GB of RAM. > It is using a Tyan Thunder K8WE (S2895A2NRF) motherboard\ > with an nVidia nForce chipset > -- http://www.tyan.com/products/html/thunderk8we.html. Yes, it is a seriously sweet board! > It is configured to boot from an SATA drive (80GB) and then > use 3Ware 8506-8 card with 8x250GB SATA drives in a big raid > 5 as a data store. Okay, someone should be shot on that one. You should be able to use the 8506-8 "out-of-the-box" for boot, no "intermedia" disk required. Option 1: *DEAD*SIMPLE*SOLUTION* YANK OUT THE SATA DISK ON THE NVIDIA CHIPSET AND USE _ONLY_ THE 3WARE CARD! NOW RE-INSTALL. Option 2: "Complex solution" (and why this happened) I'm sure part of the problem is the load order of the nv_sata and 3w-xxxx driver. If the nv_sata loads first, which uses the generic SCSI interafces, and it will be /dev/sda. The 3w-xxxx volumes will /dev/sdb on-ward. You need to tell Linux the _exact_order_ of the host adapters that will map to SCSI cards. So ... A. Decide who gets to boot first -- nVidia chipset SATA or 3Ware Escalade SATA. Then ... B. BIOS -- The S2895 uses Phoenix ServerBIOS and will let you select the _exact_ card/[S]ATA channel that gets first boot. I.e., it will even "see" the 3Ware BIOS, and list that under the Boot selection. Make sure you decide which one, and then set that in the BIOS. C. INITRD/GRUB -- You now need to setup /etc/modprobe.conf in Linux to do the same. E.g., if you select the nv_sata to boot first, /etc/modprobe.conf should have: alias scsi_hostadapter nv_sata alias scsi_hostadapter1 3w-xxxx Now you'll need to remake the "initrd-*.img" for the kernel with "mkinitrd" -- e.g., mkinitrd /boot/initrd-`uname -r`-NVfirst.img `uname -r` This will generate a new initrd file of /boot/initrd-(kernel-rev)-NVfirst.img. Now create another GRUB entry to use it and boot it. I.e., you can copy the existing entry, just change the "initrd" line to the new filename for the new entry. D. GRUB MAP/INIT -- Verify your GRUB map file (/boot/grub/device.map) says the following ... (hd0) /dev/sda (hd1) /dev/sdb [ hd2 if you have /dev/sdc, etc... ] And then run: grub-install /dev/sda NOTE: You may need to _reboot_ into the "Rescue Mode" _after_ modifying /etc/modprobe.conf so it reads the order of SCSI adapters. And even then, I'm not sure it will work, because the driver might assume the nv_sata is always first. Which is why I recommend the "dead simple solution." Install with _only_ the 3Ware Escalade attached drives, and then the installer _should_ setup /etc/modprobe.conf with "alias scsi_hostadapter 3w-xxxx" and the initrd will be correct. Also make sure you set the 3Ware to _always_ boot before the on-board ATA/SATA in the Phoenix ServerBIOS. > One oddball thing is that I cannot see the 3ware card at > all (doesn't show up in lspci output). Of course! Because it's very likely the nVidia SATA is loading _first_! That's the problem. > Any thoughts? Anyone set up a similar beastie? All-the-time. If I have a 3Ware card, there is *0* reason to use the on-board SATA channels. If you do, make sure you put the 3Ware as the _first_ card in the /etc/modprobe.conf with a "scsi_hostadapter" alias. Otherwise any other SCSI card might be assumed to be first, like the nVidia SATA with its nv_sata driver (which appears as SCSI). -- Bryan P.S. Remember, if you don't want to use RAID-5 for system volumes, you _can_ configure your 3Ware card with _multiple_ volumes. E.g., (2) RAID-1 /dev/sda for "System" (6) RAID-5 /dev/sdb for "Data" In fact, this is what I do normally. Sometimes I'll do (4) RAID-10 and (4) RAID-5, if I want some faster RAID-10 storage for some data, using the RAID-5 volume for lesser and/or more "read-only" access. -- Bryan J. Smith mailto:b.j.smith at ieee.org Sent from Yahoo Mail (please excuse any missing headers)