[CentOS] What FileSystems for large stores and very very large stores?

[Les Mikesell]

On Fri, Jul 5, 2013 at 8:45 AM, Eliezer Croitoru <eliezer at ngtech.co.il> wrote:
> I was learning about the different FS exists.
> I was working on systems that ReiserFS was the star but since there is
> no longer support from the creator there are other consolidations to be
> done.
> I want to ask about couple FS options.
> EXT4 which is amazing for one node but for more it's another story.
> I have heard about GFS2 and GlusterFS and read the docs and official
> materials from RH on them.
> In the RH docs it states the EXT4 limit files per directory is 65k and I
> had a directory which was pretty loaded with files and I am unsure
> exactly what was the size but I am almost sure it was larger the 65k
> files per directory.
>
> I was considering using GlusterFS for a very large storage system with
> NFS front.
> I am still unsure EXT4 should or shouldn't be able to handle more then
> 16TB since the linux kernel ext4 docs at:
> https://www.kernel.org/doc/Documentation/filesystems/ext4.txt in section 2.1
> it states: * ability to use filesystems > 16TB (e2fsprogs support not
> available yet).
> so can I use it or not?? if there are no tools to handle this size then
> I cannot trust it.
>
> I want to create a storage with more then 16TB based on GlusterFS since
> it allows me to use 2-3 rings FS which will allow me to put the storage
> in a form of:
> 1 client -> HA NFS servers -> GlusterFS cluster.

If you really only have one client, you might look at ceph for
distributed block storage with xfs on top so you don't need to run
through fuse.  Or if your application can be changed to use the s3
interface you could get away from the posix filesystem bottlenecks
completely.

No experience with this stuff - just sounds like the promising
up-and-coming thing...

--
   Les Mikesell
     lesmikesell at gmail.com