[CentOS] 11TB ext4 filesystem - filesystem alternatives?

Ilyas -- umask00 at gmail.com
Sat Sep 29 12:43:29 UTC 2012


XFS + battery backed RAID controller is not way to protect your data.

Very easy way to understand it is run server farm with 1000+ nodes.
This is enough quantity of servers for make representative sample.

There are problems:
1. bugs in RAID controllers (problems with BBU, cache memory,
hardware, firmware etc) which follow to errors in data writes or even
follow to freeze server which require cold reboots.
2. problems with hardware (cpu, memory, mainboard etc) which follow to
system hangups too.

Almost every system hangup makes XFS broken

In my case of problems with XFS I have 2 uninvestigated issues:
1. Why XFS zeroed files which was written and closed 2 weeks ago (it
happened on server with few terabytes mdraid1)?
2. Why xfs_check never works (even on systems with 32G of ram) on
filesystems with 40TB storage? Yes, I have to use xfs_repair, but
anyway xfs_check everytime killed by OOM even when run after
xfs_repair (this problem happened with 40TB storage with 40k files on
it). This fact forced me to store some part of backups on ext4. To do
it I have rebuilt  latest version of e2fsprogs for rhel6 because
vendor version does not support so big ext4 filesystems.





On Sat, Sep 29, 2012 at 3:30 AM, James A. Peltier <jpeltier at sfu.ca> wrote:
> ----- Original Message -----
> | Hello,
> |
> | One day our servers farm rebooted unexpectedly (power fail happened)
> | and on centos 6.3 with up2date kernel we lost few hundred files
> | (which
> | probably was opened for reading, NOT writing) on XFS.
> |
> | Unexpected power lost follow to situation when some files get a zero
> | size.
>
> This is not uncommon with a file system like XFS, where the file system makes EXTENSIVE use of file system caching and memory and internal semantics that will make your head spin.  Fact of the matter is, that in spite of this "possibility" of loss, XFS is by far the best file system for large volumes at the moment and especially during initialization time.  You *can* use EXT4 with you can speed this up if you use the -E lazy_itable_init=1 -O dir_index,extent,flex_bg,uninit_bg options.
>
> --
> James A. Peltier
> Manager, IT Services - Research Computing Group
> Simon Fraser University - Burnaby Campus
> Phone   : 778-782-6573
> Fax     : 778-782-3045
> E-Mail  : jpeltier at sfu.ca
> Website : http://www.sfu.ca/itservices
>           http://blogs.sfu.ca/people/jpeltier
>
> Success is to be measured not so much by the position that one has reached
> in life but as by the obstacles they have overcome. - Booker T. Washington
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos



-- 
GPG Key ID: 6EC5EB27



More information about the CentOS mailing list