[CentOS] NFS not recognizing available file space

[Pat Haley]

Hi James,

My system did not recognize the delaylog option,
but when I mounted with nobarrier,inode64
things worked and I was able to write to the
array!

Thanks!

Pat

> change mount to nobarrier,inode64,delaylog
> 
> ----- Original Message -----
> | 
> | Hi:
> | 
> | > On Tue, Feb 4, 2014 at 1:03 PM, Pat Haley <phaley at mit.edu> wrote:
> | > 
> | >> Hi,
> | >>
> | >> I have a server running under CentOS 5.8 and I appear to be in a
> | >> situation in which the NFS file server is not recognizing the
> | >> available
> | >> space on a particular disk (actually a hardware RAID-6 of 13 2Tb
> | >> disks).
> | >> If I try to write to the disk I get the following error message
> | >>
> | >> [root at nas-0-1 mseas-data-0-1]# touch dum
> | >> touch: cannot touch `dum': No space left on device
> | >>
> | >> However, if I check the available space, there seems to
> | >> be plenty
> | >>
> | >> [root at nas-0-1 mseas-data-0-1]# df -h .
> | >> Filesystem            Size  Used Avail Use% Mounted on
> | >> /dev/sdb1              21T   20T  784G  97% /mseas-data-0-1
> | >>
> | > 
> | > Maybe you're hitting the allocation of reserved blocks for root?
> | > With your disk usage of 97% I'd think that could be the case.
> | > 
> | > You didn't say what file system you're using for that 21TB array,
> | > so we
> | > (this list) won't be of too much help without knowing that.
> | 
> | xfs file system.  The fstab line for this array is:
> | 
> | /dev/sdb1               /mseas-data-0-1         xfs     defaults
> |        1 0
> | 
> | > 
> | > tune2fs [0] is your friend
> | 
> | if I read the man pages correctly tune2fs will not work for xfs.
> |  From xfs_info I get the following for mseas-data-0-1
> | 
> | [root at nas-0-1 mseas-data-0-1]# xfs_info .
> | meta-data=/dev/sdb1              isize=256    agcount=32,
> | agsize=167846667 blks
> |           =                       sectsz=512   attr=1
> | data     =                       bsize=4096   blocks=5371093344,
> | imaxpct=25
> |           =                       sunit=0      swidth=0 blks,
> |           unwritten=1
> | naming   =version 2              bsize=4096
> | log      =internal               bsize=4096   blocks=32768, version=1
> |           =                       sectsz=512   sunit=0 blks,
> |           lazy-count=0
> | realtime =none                   extsz=4096   blocks=0, rtextents=0
> | 
> | Unfortunately, I don't know how to interpret this or if
> | it is giving relevant information to the question at hand>
> | 
> | > - use it to determine if there are reserved blocks
> | > - use it to adjust the settings
> | > 
> | > [0]
> | > https://wiki.archlinux.org/index.php/ext4#Remove_reserved_blocks
> | > 
> | > 
> | >> [root at nas-0-1 mseas-data-0-1]# df -i .
> | >> Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> | >> /dev/sdb1            3290047552 4391552 3285656000    1%
> | >> /mseas-data-0-1
> | >>
> | >> I don't know if the following is relevant but the disk in question
> | >> is served as one of 3 bricks in a gluster namespace.
> | >>
> | >> Based on the test with touch, which is happening directly
> | >> at the NFS level, this seems to be an NFS rather than gluster
> | >> issue. I couldn't find any file in /var/log which had a
> | >> time that corresponded to the failed touch test and I didn't
> | >> see anything in dmesg.  We have tried rebooting this system.
> | >> What else should we look at and/or try to resolve or debug
> | >> this issue?
> | >>
> | > 
> | > If you have a non-root shell account on that box, can you write to
> | > that
> | > array from the NFS host?
> | > ( Take NFS out of the equation. )
> | 
> | Unfortunately we only have a root account on that box.
> | 
> | > 
> | > 
> | >> Thanks.
> | >>
> | >> Pat
> | >>
> | >> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> | >> Pat Haley                          Email:  phaley at mit.edu
> | >> Center for Ocean Engineering       Phone:  (617) 253-6824
> | >> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> | >> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> | >> 77 Massachusetts Avenue
> | >> Cambridge, MA  02139-4301
> | >> _______________________________________________
> | >> CentOS mailing list
> | >> CentOS at centos.org
> | >> http://lists.centos.org/mailman/listinfo/centos
> | >>
> | > 
> | > 
> | > 
> | 
> | 
> | --
> | 
> | -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> | Pat Haley                          Email:  phaley at mit.edu
> | Center for Ocean Engineering       Phone:  (617) 253-6824
> | Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> | MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> | 77 Massachusetts Avenue
> | Cambridge, MA  02139-4301
> | _______________________________________________
> | CentOS mailing list
> | CentOS at centos.org
> | http://lists.centos.org/mailman/listinfo/centos
> | 
> 


-- 

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-213                    http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301