[CentOS] NFS not recognizing available file space

Tue Feb 4 18:03:16 UTC 2014
Pat Haley <phaley at MIT.EDU>

Hi,

I have a server running under CentOS 5.8 and I appear to be in a
situation in which the NFS file server is not recognizing the available
space on a particular disk (actually a hardware RAID-6 of 13 2Tb disks).
If I try to write to the disk I get the following error message

[root at nas-0-1 mseas-data-0-1]# touch dum
touch: cannot touch `dum': No space left on device

However, if I check the available space, there seems to
be plenty

[root at nas-0-1 mseas-data-0-1]# df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb1              21T   20T  784G  97% /mseas-data-0-1

[root at nas-0-1 mseas-data-0-1]# df -i .
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sdb1            3290047552 4391552 3285656000    1% /mseas-data-0-1

I don't know if the following is relevant but the disk in question
is served as one of 3 bricks in a gluster namespace.

Based on the test with touch, which is happening directly
at the NFS level, this seems to be an NFS rather than gluster
issue. I couldn't find any file in /var/log which had a
time that corresponded to the failed touch test and I didn't
see anything in dmesg.  We have tried rebooting this system.
What else should we look at and/or try to resolve or debug
this issue?

Thanks.

Pat

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-213                    http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301