[CentOS] Disk array format for CentOS virtual host

Sat Dec 18 21:58:22 UTC 2010
Chuck Munro <chuckm at seafoam.net>

Hello List Members (and Holiday greetings!)

This rambles a bit ... my apologies in advance.

I am in the process of building a large CentOS-based VM host machine 
which will replace several individual boxes.  I've done the usual 
hardware research and ended up with a SuperMicro motherboard, dual 
Xeons, lots of ECC RAM, solid power supply, and tons of cooling.

When it comes to the best disk array layout for 7 guest instances of 
CentOS (mostly 64-bit) I run into the classical argument of disk I/O 
performance versus array robustness.  I'm using Linux software RAID so I 
don't get locked into a vendor's controller, and because I've got CPU 
cycles to burn.

The intent is to use CentOS-6 with KVM after climbing the usual 
experimental learning curve.  I've got years of CentOS experience, but 
none with KVM.

So, underneath all of this I could use RAID-10/RAID-6/RAID-60.  Which 
one is better for the following work mix:

  - 4 web servers, not heavily loaded (yet), mostly CGI.
  - 2 independent Zimbra email servers, which seem to spend most of 
their time rejecting spam and malware.
  - A primarily-inbound FTP server with bandwidth throttling.
  - A batch ImageMagick video process which kicks in a couple of 
times/day for a few minutes.
  - A secondary/backup SMB server, not heavily used.
  - A very lightly loaded DDNS server.

Two spindles will be used in RAID-1 for the host OS filesystems.

I have 10 large drives for the guest machines, but because they're 2 
TByte I want to ensure reasonable fault tolerance, meaning the ability 
to handle at least two drive failures.  This really means that an array 
rebuild to replace *one* drive stands a good chance of succeeding.

At first glance this would seem to be a system which will be doing more 
reads than writes, so RAID-6 or 60 might be a reasonable fit.  RAID-10 
offers much better write performance according to what I've been 
reading, but is less fault tolerant.  RAID-60 is a bit of an unknown to 
me with regard to performance.

In case you're wondering, I don't consider RAID to be a backup, so all 
of this will be automatically backed up to duplicate separate machines, 
each having tons of RAID-6 storage.  Each individual guest OS will do 
its own rsync thing.

Please forgive my lack of VM experience, but an additional question ...

I'm thinking of creating a separate RAID-xx MD partition for each guest. 
  Should I hand each over as a 'device' to KVM or is it better to place 
the guest filesystems in a big file on each separate MD partition?  Or, 
is it better to create one huge RAID-xx partition and just use big 
filesystems-in-a-file.  Either way would probably take the same amount 
of time (long!) to rebuild an array, if I understand this correctly.

The collective wisdom of this list is most welcome.

Thanks muchly,
Chuck