[CentOS] RAID 6 - opinions

Thu Apr 11 23:13:33 UTC 2013
David C. Miller <millerdc at fusion.gat.com>

----- Original Message -----
> From: "Joseph Spenner" <joseph85750 at yahoo.com>
> To: "CentOS mailing list" <centos at centos.org>
> Sent: Thursday, April 11, 2013 1:36:29 PM
> Subject: Re: [CentOS] RAID 6 - opinions
> 
> 
> 
> 
> ________________________________
>  From: John R Pierce <pierce at hogranch.com>
> To: centos at centos.org
> Sent: Thursday, April 11, 2013 1:24 PM
> Subject: Re: [CentOS] RAID 6 - opinions
>  
> 
> On 4/11/2013 12:30 PM, m.roth at 5-cent.us wrote:
> >> Ok, listening to all of this, I've also been in touch with a tech
> >> from the
> >> vendor*, who had a couple of suggestions: first, two RAID sets
> >> with two
> >> global hot spares.
> 
> 
> I would test how long a drive rebuild takes on a 20 disk RAID6.    I
> suspect, very long, like over 24 hours, assuming a fast controller
> and
> sufficient channel bandwidth.
> 

Just for reference, I have a 24 x 2TB SATAIII using CentOS 6.4 Linux MD RAID6 with two of those 24 disks as hotspares. The drives are in a Supermicro external SAS/SATA box connected to another Supermicro 1U computer with an i3-2125 CPU @ 3.30GHz and 16GB ram. The connection is via a 6Gbit mini SAS cable to an LSI 9200 HBA. Before I deployed it into production I tested how long it would take to rebuild the raid from one of the hot spares and it took a little over 9 hours. I have two 15TB LVM's on it formatted EXT4 with the rest used for LVM snapshot space if needed. Using dd to write a large file to one of the partitions I see about 480MB/s. If I rsync from one partition to another I get just under 200MB/s. 

dd if=/dev/zero of=/backup/5GB.img count=5000 bs=1M
5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 10.8293 s, 484 MB/s

David.