[CentOS] 12-15 TB RAID storage recommendations

Tue Apr 13 19:19:53 UTC 2010
Don Krause <dkrause at optivus.com>

On Apr 13, 2010, at 11:57 AM, nate wrote:

> John R Pierce wrote:
> 
>> well, IF your controller totally screams and can rebuild the drives at
>> wire speeds with full overlap, you'll be reading 7 * 2TB of data at
>> around 100MB/sec average and writing the XOR of that to the 8th drive in
>> a 8 spindle raid5 (14tb total).   just reading one drive at wirespeed is
>> 2000,000MB / 100MB == 20,000 seconds, or about 5.5 hours, so thats about
>> the shortest it possibly could be done.
> 
> More likely your looking at 24+ hours, because really no disk system
> is going to read your SATA disks at 100MB/second. If your really lucky
> perhaps you can get 10MB/second.
> 
> With the fastest RAID controllers in the industry my own storage
> array(which does heavy amounts of random I/O) averages about
> 2.2MB/second for a SATA disk, with peaks at around 4MB/second.
> 
> Our previous storage array averaged about 4-6 hours to rebuild
> a RAID 5 12+1 array with 146GB 10k RPM disks, on an array that was
> in excess of 90% idle. Rebuilding a 400GB SATA-I array often
> took upwards of 48 hours.
> 
> nate
> 
> 


For a "real life" example, we have a 3 year old 12x 1TB SATA box using an Adaptec RAID controller, doing RAID 6 that takes about 3 days to rebuild the array each time a drive fails. Which, to date, has happened 10 times... (Fortunately, this is only a BackupPC box.)

FWIW, we've not experienced a second drive failure during the rebuild process, yet. But we have had drives fail within a few weeks of each other, so it's probably going to happen one of these days..

--
Don Krause                                                                   
Head Systems Geek, 
Waver of Deceased Chickens.
Optivus Proton Therapy, Inc.
www.optivus.com
"This message represents the official view of the voices in my head."