[CentOS] Large file system idea

Mon May 19 08:52:55 UTC 2014
Andrew Holway <andrew.holway at gmail.com>

> I have not looked at Lustre, as I have heard many negative things about it
> (including Oracle ownership). The only business using Lustre where I know
> the admins has had a lot of trouble with it. No redundancy.

I know some Lustre admins that indeed have the far away stare similar to
people that have survived natural disasters. It can be somewhat unstable
and difficult to manage when you try and roll it yourself but, if you get
the professionals and have it properly supported you can have a good time.

Lustre is not owned by Oracle, its free and opensource software Licensed
under GPL v2. It does have redundancy but this is handled on the hardware
level with Active / Active object storage servers and meta data servers.

Primarily supported by Intel. Well, they have the most developers and sell
the most support contracts. It is a very interesting replacement for Hadoop
HDFS.

>
> Fhgfs looks interesting, and I am planning on looking at it, but have
> not yet done so.

The Fraunhofer Parallel Cluster File System (FhGFS) has just been spun out
of the German Institute from which is was born and has been renamed BeeGFS.
(the germans never had a knack for snappy names :).

It is a very strong contender for these kinds of workloads and is probably
just about to be fully opensourced.

In general Parallel filesystems such as Lustre are quite hard to get right
and most people fail to grasp the complexity and the skill required in
implementing them. People have a go, fsck it up (heh) and then blame the
software when it doesn't work properly. If you really have a business
requirement for insane metadata performance over single, multi petabyte
namespace you should be sure to tread lightly and carry a good support
contract.

> MooseFS and GlusterFS have both been evaluated, and were too slow. In the
> case of GlusterFS, waaaay too slow.

I believe Gluster to be a rapidly dying project however I am willing to be
set straight on this point. It seems that anyone looking at Gluster will
also be looking a Ceph and this is an obviously better system.