[CentOS] I need storage server advice

Tue May 6 19:50:04 UTC 2008
John R Pierce <pierce at hogranch.com>

Ed Morrison wrote:
> Hi:
> I need advice on implementing a storage server.  I really do not have 
> the $ to spend for a Dell iSCSI storage divice and I am thinking 
> trunning CentOS 5.x with ftp or FreeNAS.  Here is what I am looking at 
> and concerned about.
> Situation:
> My current storage needs are approximately 1.5 TB annually.  This will 
> increase to about 3.5 TB annually over the next 5 years (rough est.).  
> This box will just be a data archive and once it is full it will only 
> be used very infrequently if not used at all. Files are small up to 10 
> MB but numerous.

infrastructure to support lots of SATA drives isn't real cheap 
regardless.   you really don't want to just bolt a bunch of drives up 
inside a jumbo desktop tower and call it a server.     5 years at that 
run rate is going to be something like 12TB total storage, which using 
commodity 500GB SATA drives in raid10 will take around 48 drives.   
Thats a lot of SATA channels...

With that many spindles, you'll also want to allocate several hot spares.

I dislike raid5 for a number of reasons, and would recommend sticking 
with mirroring, eg raid1 or raid10.   You /never/ want to build a raid5 
much over about 6-8 disks, or the raid rebuild times get ridiculous and 
double drive failures will lose huge amounts of storage.

hey, have you considered the Sun x4500 ?  its a 4U(?) dual dualcore 
opteron server that comes with 48 x 500GB SATA drives. ***

> CentOS:
> Upgrading to the newer CentOS flavors.  I will not have the ability to 
> archive this data to tape and I am concerned about loosing the data 
> when upgrading the OS.  How best to handle this?
as others have said, as long as your critical data is on seperate file 
systems, there should be no issue here.

> Storage limitation.  It is my understanding that there is a 2 TB 
> storage limitation with Linux (and windows) in general particularly 
> for stability.  I see that ReiserFS can go up to 16 TB.  Is any one 
> using this?  If so, how has it been for you?

since your data is archival in nature, it really shouldn't be that hard 
to manage it as multiple 2 TB chunks on seperate file systems.   when 
you fill 2TB, take 8 x 500GB more SATA drives, raid10 them, and mount 
them as another file system, /u01, /u02, ....    keep an index file 
somewhere which logs which backups are where.

> FreeNAS
> Anyone using FreeNAS?  What is your experience?  How easy is it to add 
> new drives and keep your data?  Upgrading to newer versions?

I setup OpenFiler once, that worked quite nicely, supported NFS, SMB, 
and iSCSI, and was pretty easy to use.   I'd have to assume FreeNAS is 

*** heresy (for this list), Solaris 10, with its ZFS file system, is 
extremely good at handling very large storage configurations like this.