Quoting Mark Schoonover schoon@amgt.com:
All 9 drives are RAID5 together via the 3Ware card. Tomorrow, I'll don my earplugs and create a small 1TB RAID device to install CentOS, then RAID5 the remaining 7 drives for all the data. I'l recreate the partitions above in the 1TB array, then use LVM to add the remaining 3.5TB into the LVs I need. I could also read up on array carving, that's new to me.
I don't have any 95xx cards around, but some suggestions anyhow what I'd try if I had any. I'd try array carving first. That way you can have OS on separate LUN from the rest of your data. So it is kind of separated on hardware level. As I said before, you don't need to recompile the kernel. Just tell scsi_mod how many LUNs it needs to scan. For example, create LUN 0 to be 1 or 2 gigs, and LUN 1 to be the rest of the space. Install OS on LUN 0 (the only one it will see by default). Then use max_luns option to gain acess to LUN 1.
9-drive RAID-5 will give you 500 gig of disk space more that you would have by creating 2-drive RAID-1 and 7-drive RAID-5. So I think it is worth to try.
Even with array carving, I'd still use LVM. It might look like overkill to have volume group with single physical volume and single logical volume on it, but it will make things more managable and expandable in the future. It is trivial to setup, and overhead should be small.