[CentOS] 8-15 TB storage: any recommendations?

Tue Jan 12 18:13:08 UTC 2010
Tom Georgoulias <tomg at mcclatchyinteractive.com>

On 01/12/2010 12:20 PM, JohnS wrote:
>
> On Mon, 2010-01-11 at 16:16 -0500, Tom Georgoulias wrote:

>> CentOS 5.4 x86_64 works fine on the x4540s, I've installed it myself and
>> didn't have to do anything special to see and use all of the disks.
>>
>> In my testing, the IO was faster and the storage easier to administer
>> with when using Solaris and ZFS rather than with CentOS and software
>> raid.  That kind of box is just made for ZFS.

> Interesting, was the CentOS Box Tuned in any way?

Not really, everything was just setup using the defaults.  I had a x4540 
with 48 1TB drives and tried to recreate the raidz2 setup that I had 
tested with Solaris/zfs.  I created six 6-disk RAID6 md devices, then 
added them into a single volume group and created a huge (5TB) logical 
volume with an XFS filesystem.  I tried use one disk from each 
controller in each md device (as indicated by the hdtool that sun 
provides).  I ran a variety of tests using tools like dd, iozone, 
tiobench, and sysbench and just watched how the server behaved.
 From what I could tell, the IO wasn't evenly spread across the disks in 
the md devices, just a subset.

I probably could've tweaked it some more, but I didn't have too much 
time to spend on it at the time.  The Sun Storage 7000 series servers 
were a better fit for that project anyway.

Tom