[CentOS] Performance problems with XFS on Centos 5.4

Mon Apr 12 16:22:24 UTC 2010
Bernhard Gschaider <bgschaid_lists at ice-sf.at>

>>>>> On Fri, 9 Apr 2010 10:59:02 -0400
>>>>> "RW" == Ross Walker <rswwalker at gmail.com> wrote:

    RW> On Apr 9, 2010, at 9:59 AM, Bernhard Gschaider
    RW> <bgschaid_lists at ice-
    sf.at> wrote:

    >> Hi!
    >> During the last weeks I experienced some performance problems
    >> with a large file-system on XFS basis. Sometimes for instance
    >> ls is painfully. Immidiatly afterwards ls on the same directory
    >> is immidiate. I used strace on this ls and found that during
    >> the first ls the lstat-calls need approx 0.02s each while
    >> during the second ls the are two orders of magnitude faster.
    >> Googling around I stumbled upon some messages similar like this
    >> http://www.opensubscriber.com/message/linux-xfs@oss.sgi.com/1355060.html
    >> which have in common a) they're from around 2006 b) they
    >> suggest to increase a mount-option ihashsize. This mount option
    >> is listed as deprecated in the current kernel-doc
    >> So my question: does anyone have experience with that kind of
    >> performance problem? Do you think it is a XFS problem or are
    >> there some other tuning parameters in the kernel that could be
    >> modified for instance via /proc?
    >> The reason why I'm asking here is that it is a production
    >> file-system so I would be very unpopular if I experiment too
    >> much (a couple of reboots is OK ;) )
    >> Bernhard
    >> PS: the situation got worse during the last weeks when the
    >> file-system increased in size, so the option that some kind of
    >> buffer now is too small and I'm experiencing some kind of
    >> thrashing seems very likely to me

    RW> Are you defragging the file system regularly?

Uups. Never occured to me ("Fragmentation is soooo Windoze")
Had a look:

xfs_db> frag
actual 6349355, ideal 4865683, fragmentation factor 23.37%

This seems significant.

    RW> How much memory do you have in the system and how big is the
    RW> file system?

Memory on the system is 4Gig (2 DualCore Xenons). The filesystem is
3.5 TB of which 740 Gig are used. Which is the maximum amount used
during the one year that the filesystem is being used (that is why the
high fragmentation amazes me)

    RW> What are the XFS parameters for the file system?

Is this sufficent?

% xfs_info  /raid
meta-data=/dev/VolGroup00/LogVol05 isize=256    agcount=32, agsize=29434880 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=941916160, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal               bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

    RW> What is the storage setup?

The filesystem is on a LVM-Volume which sits on a RAID 5 (Hardware
RAID) drive

    RW> Need the info.

So the way to go forward would be using xfs_fsr on that drive. I read
some horror stories about lost files, are these to be taken seriously
(I mean they were in some Ubuntu forums ;) )

Any other thoughts on parameters?

Thanks for your time