Centos 6.2 system with xfs filesystem. I'm sharing this filesystem using nfs.
When I create a 10 gigabyte test file from a nfs client system :
dd if=/dev/zero of=10Gtest bs=1M count=10000 10000+0 records in 10000+0 records out 10485760000 bytes (10 GB) copied, 74.827 s, 140 MB/s
Output from 'ls -al ; du' during this test :
-rw-r--r-- 1 root root 429170688 Jun 8 10:13 10Gtest 654456 10Gtest
-rw-r--r-- 1 root root 1756831744 Jun 8 10:13 10Gtest 2230720 10Gtest
-rw-r--r-- 1 root root 2492145664 Jun 8 10:11 10Gtest 4348288 10Gtest
-rw-r--r-- 1 root root 4686782464 Jun 8 10:11 10Gtest 8542592 10Gtest
-rw-r--r-- 1 root root 10485760000 Jun 8 10:12 10Gtest 16943552 10Gtest
The file is using almost double the size ? Even after a few days the file is showing the same disk usage. Only umounting and remounting the filesystem fixes the problem. When I do the same test on an ext4 filesystem no issues. (same server/client)
Same issue on 2 centos 6.2 servers. I also tried to reproduce the issue on an rhel 5.8 system with xfs, but here the disk usage is ok. (I don't have a rhel6 systems with the xfs addon subscription)
Stephan
What kernel are you using ? Are you using inode64 mount option on the Cents server itself for XFS ?
What OS was the NFS client running ? 32bit ?
Just asking, as there seems to be problem with xfs on kernel 2.6.27 when using inode64 mount options regarding 32bit nfs....
Stephan van Hienen wrote:
Centos 6.2 system with xfs filesystem. I'm sharing this filesystem using nfs.
When I create a 10 gigabyte test file from a nfs client system :
dd if=/dev/zero of=10Gtest bs=1M count=10000 10000+0 records in 10000+0 records out 10485760000 bytes (10 GB) copied, 74.827 s, 140 MB/s
Output from 'ls -al ; du' during this test :
-rw-r--r-- 1 root root 429170688 Jun 8 10:13 10Gtest 654456 10Gtest
-rw-r--r-- 1 root root 1756831744 Jun 8 10:13 10Gtest 2230720 10Gtest
-rw-r--r-- 1 root root 2492145664 Jun 8 10:11 10Gtest 4348288 10Gtest
-rw-r--r-- 1 root root 4686782464 Jun 8 10:11 10Gtest 8542592 10Gtest
-rw-r--r-- 1 root root 10485760000 Jun 8 10:12 10Gtest 16943552 10Gtest
The file is using almost double the size ? Even after a few days the file is showing the same disk usage. Only umounting and remounting the filesystem fixes the problem. When I do the same test on an ext4 filesystem no issues. (same server/client)
Same issue on 2 centos 6.2 servers. I also tried to reproduce the issue on an rhel 5.8 system with xfs, but here the disk usage is ok. (I don't have a rhel6 systems with the xfs addon subscription)
Stephan _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Mon, 11 Jun 2012, Svavar Örn Eysteinsson wrote:
What kernel are you using ?
Latest centos 6.2 kernel: 2.6.32-220.17.1.el6.x86_64
Are you using inode64 mount option on the Cents server itself for XFS ?
fstab for this filesystem :
/dev/mapper/VolGroup01-LogVol00 /raid xfs defaults 1 2
/etc/exports for this filesystem :
/raid 192.168.178.5(rw,no_root_squash)
and on the client system /etc/fstab :
server:/raid /raid nfs soft,rsize=32768,wsize=32768,vers=3
What OS was the NFS client running ? 32bit ?
Also centos 6.2 x64 with kernel 2.6.32-220.17.1.el6.x86_64.
I also have this problem with a nfs client running 2.6.18-7.4-dm8000 (dreambox satelite receiver) And Centos 5.8 x64 with kernel 2.6.18-308.8.1.el5
Stephan
On Jun 11, 2012, at 4:47 AM, Stephan van Hienen wrote:
On Mon, 11 Jun 2012, Svavar Örn Eysteinsson wrote:
What kernel are you using ?
Latest centos 6.2 kernel: 2.6.32-220.17.1.el6.x86_64
Are you using inode64 mount option on the Cents server itself for XFS ?
fstab for this filesystem :
/dev/mapper/VolGroup01-LogVol00 /raid xfs defaults 1 2
/etc/exports for this filesystem :
/raid 192.168.178.5(rw,no_root_squash)
and on the client system /etc/fstab :
server:/raid /raid nfs soft,rsize=32768,wsize=32768,vers=3
What OS was the NFS client running ? 32bit ?
Also centos 6.2 x64 with kernel 2.6.32-220.17.1.el6.x86_64.
I also have this problem with a nfs client running 2.6.18-7.4-dm8000 (dreambox satelite receiver) And Centos 5.8 x64 with kernel 2.6.18-308.8.1.el5
Hi Stephan,
I also run 6.2 with XFS but am getting normal behavior.
I ran your exact command and du -hs shows 9.8GB used.
And ls -l shows 10485760000
Would you like more info on my system?
- aurf
On Mon, 11 Jun 2012, aurfalien wrote:
Hi Stephan,
I also run 6.2 with XFS but am getting normal behavior.
I ran your exact command and du -hs shows 9.8GB used.
And ls -l shows 10485760000
Would you like more info on my system?
aurf,
Any updates not installed on your system ? (older kernel maybe?) And what size is your xfs filesystem (I tested the issue on 2.5tb, 150gb and a 5t xfs filesystem)
Stephan
On Jun 11, 2012, at 10:39 AM, Stephan van Hienen wrote:
On Mon, 11 Jun 2012, aurfalien wrote:
Hi Stephan,
I also run 6.2 with XFS but am getting normal behavior.
I ran your exact command and du -hs shows 9.8GB used.
And ls -l shows 10485760000
Would you like more info on my system?
aurf,
Any updates not installed on your system ? (older kernel maybe?) And what size is your xfs filesystem (I tested the issue on 2.5tb, 150gb and a 5t xfs filesystem)
Stephan
Hi,
My system info;
kernel 2.6.32-220.17.1.el6.x86_64 xfsprogs-3.1.1-6.el6.x86_64
Mu system is up to dat as of today. I'm pretty good at checking updates on a weekly basis.
I have chosen not to use LVM with my 20TB Raid 10 XFS file system.
I read some were that potential for problems exist due to UUID conflicts.
At any rate, I do not know how to get specifics on exactly what version of XFS that my file system is but this was a from scratch 6.2 install.
- aurf
Stephan van Hienen wrote:
The file is using almost double the size ? Even after a few days the file is showing the same disk usage. Only umounting and remounting the filesystem fixes the problem. When I do the same test on an ext4 filesystem no issues. (same server/client)
What does:
/usr/sbin/xfs_bmap -pl 10Gtest
output (when run on the server)?
You can also 'recover' the lost space by running (as root on the server):
/usr/sbin/xfs_fsr 10Gtest
James Pearson
P.S. you can also see the allocated size of a file using the -s option to ls
On Mon, 11 Jun 2012, James Pearson wrote:
What does:
/usr/sbin/xfs_bmap -pl 10Gtest
output (when run on the server)?
10Gtest: 0: [0..808319]: 261386472..262194791 808320 blocks 1: [808320..1357951]: 273699584..274249215 549632 blocks 2: [1357952..2406527]: 307386624..308435199 1048576 blocks 3: [2406528..2631927]: 308443904..308669303 225400 blocks 4: [2631928..4728959]: 1279691136..1281788167 2097032 blocks 5: [4728960..7704575]: 5285370624..5288346239 2975616 blocks 6: [7704576..8928511]: 5333860864..5335084799 1223936 blocks 7: [8928512..11709047]: 5219053184..5221833719 2780536 blocks 8: [11709048..14365431]: 3943770240..3946426623 2656384 blocks 9: [14365432..16899703]: 4437210880..4439745151 2534272 blocks 10: [16899704..17327103]: 4449971968..4450399367 427400 blocks 11: [17327104..18992639]: 3578345984..3580011519 1665536 blocks 12: [18992640..20356767]: 4200870752..4202234879 1364128 blocks 13: [20356768..20479999]: 4833581696..4833704927 123232 blocks
You can also 'recover' the lost space by running (as root on the server):
/usr/sbin/xfs_fsr 10Gtest
doesn't help :
]# xfs_fsr 10Gtest ]# du -hs 17G .
I'm also running xfs_fsr /dev/VolGroup01/LogVol00 weekly, which doesn't help.
P.S. you can also see the allocated size of a file using the -s option to ls
Thanks, this shows the same info as du.
Stephan
Stephan van Hienen wrote:
You can also 'recover' the lost space by running (as root on the server):
/usr/sbin/xfs_fsr 10Gtest
doesn't help :
]# xfs_fsr 10Gtest ]# du -hs 17G .
... it worked for me :-)
Interestingly, I did a similar 'dd' locally on a XFS file system (CentOS 6.2) and got an allocation size of 16Gb for a 10Gb file - may be it's a bug/feature of the XFS versions used?
I guess you might get a better response from the XFS mail list - see http://oss.sgi.com/projects/xfs/ ???
James Pearson
On Mon, Jun 11, 2012 at 05:07:05PM +0200, Stephan van Hienen wrote:
On Mon, 11 Jun 2012, James Pearson wrote:
What does:
/usr/sbin/xfs_bmap -pl 10Gtest
output (when run on the server)?
10Gtest: 0: [0..808319]: 261386472..262194791 808320 blocks
...
You can also 'recover' the lost space by running (as root on the server):
/usr/sbin/xfs_fsr 10Gtest
doesn't help :
]# xfs_fsr 10Gtest ]# du -hs 17G .
try mounting the xfs volume with allocsize=4k.
Tru
-
aurfalien -
James Pearson -
Stephan van Hienen -
Svavar Örn Eysteinsson -
Tru Huynh