[CentOS] 8-15 TB storage: any recommendations?

Fri Jan 8 01:01:26 UTC 2010
Christopher Chan <christopher.chan at bradbury.edu.hk>

Warren Young wrote:
> On 1/6/2010 2:35 PM, Boris Epstein wrote:
>> we are trying to set
>> up some storage servers to run under Linux
> 
> You should also consider FreeBSD 8.0, which has the newest version of 
> ZFS up and running stably on it.  I use Linux for most server tasks, but 
> for big storage, Linux just doesn't have anything like this yet.


http://security.freebsd.org/advisories/FreeBSD-SA-10:03.zfs.asc

Nothing really big but it does kinda leave doubts...interesting that 
FreeBSD has absorbed pf and zfs and now claims to be twice as fast as 
Linux for mysql/postgresql workloads. Certainly sounds very different 
from the FreeBSD 4.4 that I knew.

FreeBSD may now have ZFS support but it does not look quite the same as 
it does on Solaris/OpenSolaris.

> 
> I'm not recommending OpenSolaris on purpose.  For the last few years, it 
> was the only stable production-quality implementation of ZFS, but with 
> FreeBSD 8.0, it just lost that advantage.  I think, as a Linux fan, you 
> will be happier with FreeBSD than OpenSolaris.

Serious system administrators are not Linux fans I don't think. I tend 
to want to use the right tool for the job like OpenBSD for firewalling 
for example. I don't know about you but I find pkg on OpenSolaris to be 
more akin to yum or apt than ports and then there is always nexenta if I 
really want a complete GNU userland and apt/dpkg. I could not find out 
much are ZFS on FreeBSD. Its  man page is just a copy of the Solaris 
one. Does it support direct sharing/exporting as nfs/cifs/iscsi like it 
does on Solaris/OpenSolaris? Does it support using ZFS for booting and 
boot environments and a related upgrade system?

Nice that FreeBSD has improved its zfs support, I remember one person 
dissing zfs and pointing to vinum as an alternative but then maybe he 
did not know what he was talking about. However, there certainly is a 
lot more on vinum than there is on zfs in the FreeBSD manual.

> 
>> storage volume would be in the range specified: 8-15 TB.
> 
> That puts you right on the edge of workability with 32-bit hardware. 
> ext3's limit on 32-bit is 8 TB, and you can push it to 16 TB by 
> switching to XFS or JFS.  Best to use 64-bit hardware if you can.

Probably XFS if you want data guarantees on anything that is not a 
hardware raid card with bbu cache since JFS does not support barriers yet.