On Tue, Dec 11, 2012 at 4:01 PM, Steve Thompson smt@vgersoft.com wrote:
This is in fact a very interesting question. The default value of RPCNFSDCOUNT (8) is in my opinion way too low for many kinds of NFS servers. My own setup has 7 NFS servers ranging from small ones (7 TB disk served) to larger ones (25 TB served), and there are about 1000 client cores making use of this. After spending some time looking at NFS performance problems, I discovered that the number of nfsd's had to be much higher to prevent stalls. On the largest servers I now use 256-320 nfsd's, and 64 nfsd's on the very smallest ones. Along with suitable adjustment of vm.dirty_ratio and vm.dirty_background_ratio, this makes a huge difference.
Could you perhaps elaborate a bit on your scenario? In particular, how much memory and CPU cores do the servers have with the really high NFSD counts? Is there a rule of thumb for nfsd counts relative to the system specs? Or, like so many IO tuning situations, just a matter of "test and see"?