[CentOS] 40TB File System Recommendations

[Ross Walker]

On Apr 12, 2011, at 8:53 AM, Rudi Ahlers <Rudi at SoftDux.com> wrote:

> On Tue, Apr 12, 2011 at 2:47 PM, Marian Marinov <mm at yuhu.biz> wrote:
>> 
>> Steve,
>> I'm managing machines with 30TB of storage for more then two years. And with
>> good reporting and reaction we have never had to run fsck.
>> 
>> However I'm sure that if you have to run fsck on so big file systems, it will
>> be fater to rebuild the array from other storage then waiting for a few weeks
>> to finish.
>> 
>> On machines like that I use CentOS but I'm pratitioning them before the
>> install with a rescue live cd that I have created for me.
>> 
>> Marian
>> 
>> _______________________________________________
> 
> As matter of interest, what hardware do you use? i.e. what CPU's, size
> of RAM and RAID cards do you use on this size system?
> 
> Everyone always recommends to use smaller RAID arrays than one big fat
> one. So, I'm interested to know what you use, and how effective it
> works. i.e. if that 30TB was actively used by many hosts how does it
> cope? Or is it just archival storage?

I would never create a RAID5/6 greater then 8 disks. Usually I create a 6 or 7 disk RAID5 which means I can fit 2 in a 15 disk enclosure and have a hot spare and stripe them.

The more RAID5 sets you have the greater the write IOPS you can achieve.

Though for max IOPS nothing beats RAID10.

-Ross