[CentOS] NFS subdirectory on client is out of sync

Wed May 14 13:28:59 UTC 2008
Theo Band [GreenPeak] <theo.band at greenpeak.com>

Today a user asked me whether a file on one host can be different on 
another host. I was busy composing an answer to tell that the /home 
space on all clients are mounted using NFS from the file server. Any 
host will therefor see the same file. The user pointed me to his file 
and I copied this file from the client and compared this with the file 
on the file server. To my surprise it turned out that he was right, the 
files were different. I created a new file in this directory and it was 
not created on the file server. I renamed the file, and that was only 
seen on this single client. How can this happen?

My setup
file server (arend)
CentOS release 4.6
# grep /home /etc/exports
/home                    *(rw,sync,no_subtree_check)

On the clients (also CentOS release 4.6) I mount /home with these options:
arend:/home             /home            nfs     
proto=tcp,nfsvers=3,bg,defaults        0 0

To debug I created (su stbo) on the client small test files (touch test) 
in each directory all the way to the user /home dir. It turns out that 
one subdirectory and everything below was not synchronized to the 
server. I could create files, move them, but it was just as if I was 
working on a local disk. Other users did not experience any problem on 
this machine so it was only one sub-directory (and everything below).

I checked the syslog both on the client and on the server, but no 
messages of interest.


[root at arend ~]# stat 
/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd
  File: `/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd'
  Size: 53214           Blocks: 112        IO Block: 4096   regular file
Device: fd01h/64769d    Inode: 6614395     Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  635/    stbo)   Gid: (  635/    stbo)
Access: 2008-05-14 12:46:34.000000000 +0200
Modify: 2008-05-14 10:08:07.000000000 +0200
Change: 2008-05-14 10:08:07.000000000 +0200

I renamed the filename on the client and did stat there as well. The 
modify time shows this file is indeed older as the user mentioned.
[stbo at junco /root]$stat 
/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd_theo_test
  File: 
`/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd_theo_test'
  Size: 53214           Blocks: 112        IO Block: 32768  regular file
Device: 14h/20d Inode: 6583089     Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  635/    stbo)   Gid: (  635/    stbo)
Access: 2008-05-14 12:47:07.000000000 +0200
Modify: 2008-04-09 13:09:23.000000000 +0200
Change: 2008-05-14 12:24:24.000000000 +0200

After rebooting everything is normal again:
[root at junco ~]# stat 
/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd
  File: `/home/stbo/workarea/toekan/design/dig/vhdl/fpga/fpga_top.vhd'
  Size: 53214           Blocks: 112        IO Block: 32768  regular file
Device: 14h/20d Inode: 6614395     Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  635/    stbo)   Gid: (  635/    stbo)
Access: 2008-05-14 12:46:34.000000000 +0200
Modify: 2008-05-14 10:08:07.000000000 +0200
Change: 2008-05-14 10:08:07.000000000 +0200


Any clue what could have gone wrong? Since I trust on a working NFS, I 
like to understand what could have gone wrong. Any suggestions are welcome.


Thanks,
Theo