[CentOS] 12-15 TB RAID storage recommendations

[Seth Bardash]

Just finished building a new server for inhouse use.   12.8TB

Needed a nfs and samba server which could store 10TB and be reliable.
Also wanted to replace out DHCP server and our internal DNS server.
So needed to run dnsmasq plus ntp for time serving.

This machine replace 3 older units.

We used a Supermicro 3U case with 15 sata shuttles and 2N+1 760 Watt PS.
On ebay for 300.00

Used a Supermicro H8DME-2 MB cause I had one sitting around.
If I was going to buy one it would have been a Tyan S2932G2NR-SI for 
reliability (we have sold a bunch of these and they always work)

Installed dual AMD 2382's and 16 GB DDR2-800 RECC plus 4 wire HSF's.

Went to the 3ware online store and purchased a 9550SXU-16ML with the 
breakout cables for 340.00  Yes, I know its PCI-X. It was inexpensive 
and it runs almost as fast as the 9650 series at twice the price.

Installed 15 pcs of the WD1001FALS disks. Very reliable and low cost 
even though they are desktop drives. Used these since we always have 3 
on the shelf as spares and WD turns a bad one around in about 5 days. We 
have installed over 200 of these drives in raid arrays and have had 2 
fail in the last 6 months.

BTW, Initializing took <8 hours not days. Set it up as a Raid 5 array 
with one hot spare and autorebuild. Have tested it with both Centos 5.4 
x86_64 and OpenSUSE 11.1 x86_64. Also installed an LSI SCSI card and a 
Sony AIT-5 Tape drive with Bacula for data backup and recovery. Again, 
all worked fine and the machine was able to do backups and keep a GigE 
connection saturated.

Using an in house piece of code called disktest we are seeing 268 MB / 
sec on writes and 347 MB / sec on reads using 32 GB test files, 131K 
buffer sizes and single threading.

If you would like to see a system that is almost identical, have a look 
at Coraid.com. They use a dual xeon and a BSD custom kernel.

Hope this config helps......

Seth Bardash

Integrated Solutions and Systems LLC
seth at integratedsolutions.org
Failure cannot survive knowledge and perseverance!