[CentOS] RAID 6 - opinions

[David C. Miller]

----- Original Message -----
> From: "Keith Keller" <kkeller at wombat.san-francisco.ca.us>
> To: centos at centos.org
> Sent: Thursday, April 11, 2013 4:34:20 PM
> Subject: Re: [CentOS] RAID 6 - opinions
> 
> On 2013-04-11, David C. Miller <millerdc at fusion.gat.com> wrote:
> >
> > Just for reference, I have a 24 x 2TB SATAIII using CentOS 6.4
> > Linux MD RAID6 with two of those 24 disks as hotspares. The drives
> > are in a Supermicro external SAS/SATA box connected to another
> > Supermicro 1U computer with an i3-2125 CPU @ 3.30GHz and 16GB ram.
> > The connection is via a 6Gbit mini SAS cable to an LSI 9200 HBA.
> > Before I deployed it into production I tested how long it would
> > take to rebuild the raid from one of the hot spares and it took a
> > little over 9 hours.
> 
> I did a similar test on a 3ware controller.  Apparently those cards
> have
> a feature that allows the controller to remember which sectors on the
> disks it has written, so that on a rebuild it only reexamines those
> sectors.  This greatly reduces rebuild time on a mostly empty array,
> but
> it means that a good test would almost fill the array, then attempt a
> rebuild.  I definitely saw a difference in rebuild times as I filled
> the
> array.  (In 3ware/LSI world this is sometimes called "rapid RAID
> recovery".)
> 
> In checking my archives, it looks like a rebuild on an almost full
> 50TB
> array (24 disks) took about 16 hours.  That's still pretty
> respectable.
> I didn't repeat the experiment, unfortunately.
> 
> I don't know if your LSI controller has a similar feature, but it's
> worth investigating.
> 
> --keith
> 

The LSI 9200's I use are nothing more than a dumb $300 host bus adapter. No RAID levels or special features. I prefer to NOT use hardware RAID controllers when I can. With a generic HBA the hard drives are seen raw to the OS. You can use smartctl to poll and test the drives just like they were connected to a generic SATA bus on the motherboard. The tools built into Linux(smartd & md) are better suited and more flexible at reporting problems and handling every level of RAID. It also makes migrating the array to another system trivial. I don't have to worry about finding the exact same RAID controller. Just a no frills SAS/SATA HBA will do.

David.