[CentOS] Too much cpu wait on nfs server when we need to read data on it

Mon Jan 25 13:39:47 UTC 2010
fabien faye <fabien at faye.eu>

Hi,

I have a big server with 24 Disk on 2 3ware card.

When i write data on my nfs server everything is fine but when i want to read data i have a lot of cpu wait.

[root at NFS /]# vmstat 1
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  1    112   9592   7140 1879496    0    0   621  1180    6    2  0  2 86 11  0
 0  9    112  10668   7148 1877760    0    0    84 21600 1926  454  0  3 49 49  0
 1  3    112   9308   7156 1879092    0    0  1400  1580 2123 2009  0  2 50 48  0
 0  0    112   9432   7156 1879476    0    0  3664     0 2275 2862  0  3 81 17  0
 0  1    112   9212   7172 1879588    0    0  2196    32 2132 2293  0  2 66 32  0
 1  0    112   9848   7180 1878864    0    0  1268  4168 1852 1934  0  2 84 14  0
 0  7    112  11208   7180 1876936    0    0    92 13124 1339  355  0  2 50 48  0
 0  0    112  10708   7188 1877512    0    0   628    16 1552 1014  0  1 84 16  0
 0  1    112  10344   7188 1878044    0    0   472     0 1334  721  0  1 89 10  0
 0  1    112  10372   7196 1878136    0    0  1936    32 2268 2850  0  2 50 48  0
 0  0    112  10336   7192 1878312    0    0  2220     0 2308 2966  0  2 51 47  0
 0  0    112   9716   7200 1879012    0    0   696    28 1484  885  0  1 89 11  0
 0  2    112   9400   7200 1879280    0    0   236     0 2112 5819  0  8 75 18  0
 0  0    112   9980   7196 1878816    0    0  1360    20 1840 1743  0  1 83 15  0
 0 15    112   9168   7196 1879580    0    0   768     0 1423  728  0  1 91  9  0
 0  0    112  10756   7200 1877752    0    0  4068    32 2788 4225  0  4 70 26  0

I have tuned nfs server with this option :

RPCNFSDARGS="-N 2"
RPCNFSDARGS="-N 4"
MOUNTD_NFS_V3="yes"
RPCNFSDCOUNT=160

about the file system, i use ext4 on LVM partition.

Do you have any idea about that.

Fabien FAYE
RHCE
www.generationip.com
Free network tools & HOWTO for centos and Redhat