[CentOS] 12-15 TB RAID storage recommendations

Tue Apr 13 19:24:56 UTC 2010
Drew Weaver <drew.weaver at thenap.com>

Those drives are likely fading out of the array because they aren't meant to be in arrays in the first place, Adaptec has told us that if you use consumer drives with their cards you are operating at your own risk.


-----Original Message-----
From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On Behalf Of Don Krause
Sent: Tuesday, April 13, 2010 3:20 PM
To: CentOS mailing list
Subject: Re: [CentOS] 12-15 TB RAID storage recommendations

On Apr 13, 2010, at 11:57 AM, nate wrote:

> John R Pierce wrote:
>> well, IF your controller totally screams and can rebuild the drives at
>> wire speeds with full overlap, you'll be reading 7 * 2TB of data at
>> around 100MB/sec average and writing the XOR of that to the 8th drive in
>> a 8 spindle raid5 (14tb total).   just reading one drive at wirespeed is
>> 2000,000MB / 100MB == 20,000 seconds, or about 5.5 hours, so thats about
>> the shortest it possibly could be done.
> More likely your looking at 24+ hours, because really no disk system
> is going to read your SATA disks at 100MB/second. If your really lucky
> perhaps you can get 10MB/second.
> With the fastest RAID controllers in the industry my own storage
> array(which does heavy amounts of random I/O) averages about
> 2.2MB/second for a SATA disk, with peaks at around 4MB/second.
> Our previous storage array averaged about 4-6 hours to rebuild
> a RAID 5 12+1 array with 146GB 10k RPM disks, on an array that was
> in excess of 90% idle. Rebuilding a 400GB SATA-I array often
> took upwards of 48 hours.
> nate

For a "real life" example, we have a 3 year old 12x 1TB SATA box using an Adaptec RAID controller, doing RAID 6 that takes about 3 days to rebuild the array each time a drive fails. Which, to date, has happened 10 times... (Fortunately, this is only a BackupPC box.)

FWIW, we've not experienced a second drive failure during the rebuild process, yet. But we have had drives fail within a few weeks of each other, so it's probably going to happen one of these days..

Don Krause                                                                   
Head Systems Geek, 
Waver of Deceased Chickens.
Optivus Proton Therapy, Inc.
"This message represents the official view of the voices in my head."

CentOS mailing list
CentOS at centos.org