hi all, I've done more investigation into this problem. I think I uncovered a bug, but I want a sanity check... (and please excuse me for top posting - I wanted to include the original message for easier reference).

As a reminder, I have a XFS filesystem exported via nfs4. The nfs4 clients (all 13) report the quota incorrectly for a user with >6TB data. On the NFS server, the 'quota' and 'xfs_quota' commands report disk usage correctly.

I modified the this user's quota on the server, and then report the output of the 'quota' command below. I start with a quota equal to the disk usage (6635021816), and then decrease it, to the point that the client reports the same quota value as the server.

Note: the lines separate instances where I adjusted the quota, where the top line is 'quota' output on the nfs server, the second line is from the nfs client. (The asterisk just indicates that the user exceeded the quota block limit).

----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816 6635021816

Filesystem blocks quota server:/fs 2340054520 2340054520 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294990000

Filesystem blocks quota server:/fs 2340054520* 22704 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294970000

Filesystem blocks quota server:/fs 2340054520* 2704 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294970000

Filesystem blocks quota server:/fs 2340054520* 1704 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294968000

Filesystem blocks quota server:/fs 2340054520* 704 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294967900

Filesystem blocks quota server:/fs 2340054520* 604 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294967500

Filesystem blocks quota server:/fs 2340054520* 204 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294967300

Filesystem blocks quota server:/fs 2340054520* 4 ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294967296

<NONE> ----------------------------------------- Filesystem blocks quota /dev/mapper/fs 6635021816* 4294967292

Filesystem blocks quota server:/fs 2340054520 4294967292 -----------------------------------------

I think it's interesting that the second to last instance didn't report any quota for this user on the client. Of course, increasing or decreasing the quota produced output.

Any insight?

Thanks! Julie

On 24-06-2011 17.17 -0700, Julie Ashworth wrote:

hi all, I hope you can help me with a strange quota/nfs/XFS behavior...

I'm using Centos 5.6 on both a nfs server and client. Both are 64-bit, and using a recent kernel:

NFS server: Linux fs2.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

NFS client: Linux nx8.priv 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

The NFS server is exporting a XFS filesystem: Filesystem Size Used Avail Use% Mounted on /dev/mapper/vgXX-lvXX 24T 16T 8.2T 66% /export

User foo (for anonymity) added ~3TB of data in the last 2-3 days. On the NFS server, her quota is reported as ~5.8 TB:

Disk quotas for user foo (uid 1314): Filesystem blocks quota limit grace files quota limit grace /dev/mapper/vgXXX-lvXXX 6129681348 7490000000 0 6991914 0 0

On a NFS client, her quota is reported as ~1.8TB:

Disk quotas for user foo (uid 1314): Filesystem blocks quota limit grace files quota limit grace server:/XXXXX 1831504512 3195032704 0 6988049 0 0

When I modify her quota on the NFS sever, it's not reflected on the NFS client.

This problem is specific user foo. There are about 20 other users with quotas on this NFS server, and if I change one of their quotas, the quota/usage is correctly (and instantly) reflected on the NFS client.

rpc.quotad is configured to use port 875, and I can see this is working with strace. And tcpdump shows an established connection: 16:50:04.480757 IP nx8.priv.pssc > fs2.priv.875: UDP, length 116 16:50:04.481015 IP fs2.priv.875 > nx8.priv.pssc: UDP, length 68

Any ideas?

Thanks! Best, Julie

-- Julie Ashworth julie.ashworth@berkeley.edu http://www.neuro.berkeley.edu PGP Key ID: 0x17F013D2 _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

---end quoted text---