Hi: > On Tue, Feb 4, 2014 at 1:03 PM, Pat Haley <phaley at mit.edu> wrote: > >> Hi, >> >> I have a server running under CentOS 5.8 and I appear to be in a >> situation in which the NFS file server is not recognizing the available >> space on a particular disk (actually a hardware RAID-6 of 13 2Tb disks). >> If I try to write to the disk I get the following error message >> >> [root at nas-0-1 mseas-data-0-1]# touch dum >> touch: cannot touch `dum': No space left on device >> >> However, if I check the available space, there seems to >> be plenty >> >> [root at nas-0-1 mseas-data-0-1]# df -h . >> Filesystem Size Used Avail Use% Mounted on >> /dev/sdb1 21T 20T 784G 97% /mseas-data-0-1 >> > > Maybe you're hitting the allocation of reserved blocks for root? > With your disk usage of 97% I'd think that could be the case. > > You didn't say what file system you're using for that 21TB array, so we > (this list) won't be of too much help without knowing that. xfs file system. The fstab line for this array is: /dev/sdb1 /mseas-data-0-1 xfs defaults 1 0 > > tune2fs [0] is your friend if I read the man pages correctly tune2fs will not work for xfs. From xfs_info I get the following for mseas-data-0-1 [root at nas-0-1 mseas-data-0-1]# xfs_info . meta-data=/dev/sdb1 isize=256 agcount=32, agsize=167846667 blks = sectsz=512 attr=1 data = bsize=4096 blocks=5371093344, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 Unfortunately, I don't know how to interpret this or if it is giving relevant information to the question at hand> > - use it to determine if there are reserved blocks > - use it to adjust the settings > > [0] https://wiki.archlinux.org/index.php/ext4#Remove_reserved_blocks > > >> [root at nas-0-1 mseas-data-0-1]# df -i . >> Filesystem Inodes IUsed IFree IUse% Mounted on >> /dev/sdb1 3290047552 4391552 3285656000 1% /mseas-data-0-1 >> >> I don't know if the following is relevant but the disk in question >> is served as one of 3 bricks in a gluster namespace. >> >> Based on the test with touch, which is happening directly >> at the NFS level, this seems to be an NFS rather than gluster >> issue. I couldn't find any file in /var/log which had a >> time that corresponded to the failed touch test and I didn't >> see anything in dmesg. We have tried rebooting this system. >> What else should we look at and/or try to resolve or debug >> this issue? >> > > If you have a non-root shell account on that box, can you write to that > array from the NFS host? > ( Take NFS out of the equation. ) Unfortunately we only have a root account on that box. > > >> Thanks. >> >> Pat >> >> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- >> Pat Haley Email: phaley at mit.edu >> Center for Ocean Engineering Phone: (617) 253-6824 >> Dept. of Mechanical Engineering Fax: (617) 253-8125 >> MIT, Room 5-213 http://web.mit.edu/phaley/www/ >> 77 Massachusetts Avenue >> Cambridge, MA 02139-4301 >> _______________________________________________ >> CentOS mailing list >> CentOS at centos.org >> http://lists.centos.org/mailman/listinfo/centos >> > > > -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Pat Haley Email: phaley at mit.edu Center for Ocean Engineering Phone: (617) 253-6824 Dept. of Mechanical Engineering Fax: (617) 253-8125 MIT, Room 5-213 http://web.mit.edu/phaley/www/ 77 Massachusetts Avenue Cambridge, MA 02139-4301