change mount to nobarrier,inode64,delaylog ----- Original Message ----- | | Hi: | | > On Tue, Feb 4, 2014 at 1:03 PM, Pat Haley <phaley at mit.edu> wrote: | > | >> Hi, | >> | >> I have a server running under CentOS 5.8 and I appear to be in a | >> situation in which the NFS file server is not recognizing the | >> available | >> space on a particular disk (actually a hardware RAID-6 of 13 2Tb | >> disks). | >> If I try to write to the disk I get the following error message | >> | >> [root at nas-0-1 mseas-data-0-1]# touch dum | >> touch: cannot touch `dum': No space left on device | >> | >> However, if I check the available space, there seems to | >> be plenty | >> | >> [root at nas-0-1 mseas-data-0-1]# df -h . | >> Filesystem Size Used Avail Use% Mounted on | >> /dev/sdb1 21T 20T 784G 97% /mseas-data-0-1 | >> | > | > Maybe you're hitting the allocation of reserved blocks for root? | > With your disk usage of 97% I'd think that could be the case. | > | > You didn't say what file system you're using for that 21TB array, | > so we | > (this list) won't be of too much help without knowing that. | | xfs file system. The fstab line for this array is: | | /dev/sdb1 /mseas-data-0-1 xfs defaults | 1 0 | | > | > tune2fs [0] is your friend | | if I read the man pages correctly tune2fs will not work for xfs. | From xfs_info I get the following for mseas-data-0-1 | | [root at nas-0-1 mseas-data-0-1]# xfs_info . | meta-data=/dev/sdb1 isize=256 agcount=32, | agsize=167846667 blks | = sectsz=512 attr=1 | data = bsize=4096 blocks=5371093344, | imaxpct=25 | = sunit=0 swidth=0 blks, | unwritten=1 | naming =version 2 bsize=4096 | log =internal bsize=4096 blocks=32768, version=1 | = sectsz=512 sunit=0 blks, | lazy-count=0 | realtime =none extsz=4096 blocks=0, rtextents=0 | | Unfortunately, I don't know how to interpret this or if | it is giving relevant information to the question at hand> | | > - use it to determine if there are reserved blocks | > - use it to adjust the settings | > | > [0] | > https://wiki.archlinux.org/index.php/ext4#Remove_reserved_blocks | > | > | >> [root at nas-0-1 mseas-data-0-1]# df -i . | >> Filesystem Inodes IUsed IFree IUse% Mounted on | >> /dev/sdb1 3290047552 4391552 3285656000 1% | >> /mseas-data-0-1 | >> | >> I don't know if the following is relevant but the disk in question | >> is served as one of 3 bricks in a gluster namespace. | >> | >> Based on the test with touch, which is happening directly | >> at the NFS level, this seems to be an NFS rather than gluster | >> issue. I couldn't find any file in /var/log which had a | >> time that corresponded to the failed touch test and I didn't | >> see anything in dmesg. We have tried rebooting this system. | >> What else should we look at and/or try to resolve or debug | >> this issue? | >> | > | > If you have a non-root shell account on that box, can you write to | > that | > array from the NFS host? | > ( Take NFS out of the equation. ) | | Unfortunately we only have a root account on that box. | | > | > | >> Thanks. | >> | >> Pat | >> | >> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- | >> Pat Haley Email: phaley at mit.edu | >> Center for Ocean Engineering Phone: (617) 253-6824 | >> Dept. of Mechanical Engineering Fax: (617) 253-8125 | >> MIT, Room 5-213 http://web.mit.edu/phaley/www/ | >> 77 Massachusetts Avenue | >> Cambridge, MA 02139-4301 | >> _______________________________________________ | >> CentOS mailing list | >> CentOS at centos.org | >> http://lists.centos.org/mailman/listinfo/centos | >> | > | > | > | | | -- | | -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- | Pat Haley Email: phaley at mit.edu | Center for Ocean Engineering Phone: (617) 253-6824 | Dept. of Mechanical Engineering Fax: (617) 253-8125 | MIT, Room 5-213 http://web.mit.edu/phaley/www/ | 77 Massachusetts Avenue | Cambridge, MA 02139-4301 | _______________________________________________ | CentOS mailing list | CentOS at centos.org | http://lists.centos.org/mailman/listinfo/centos | -- James A. Peltier Manager, IT Services - Research Computing Group Simon Fraser University - Burnaby Campus Phone : 778-782-6573 Fax : 778-782-3045 E-Mail : jpeltier at sfu.ca Website : http://www.sfu.ca/itservices "I want to inspire people. I want someone to say "because of you I didn't give up". - Chanda Kaushik