[CentOS] odd inconsistency with nfs

Fri Jul 5 21:54:40 UTC 2013
Miranda Hawarden-Ogata <hawarden at ifa.hawaii.edu>

James H, James P, and Leon, thanks for your replies. Here's what I have 
in answer to your questions/suggestions:

We do not use tcp_wrappers or iptables. The functional server is the 
redhat 7.3 box, the semi-functional server is the centos 6.4 box. On the 
centos 6.4 box, one share works as expected and the other does not. The 
use of symlinks does not appear to matter as both shares use them and 
one works while the other does not.

For now, I am ignoring automounting and mounting everything manually so 
that I can explicitly declare the options I want each time.

A summary of the testing (detailed output below):

The two shares on the centos 6.4 server have been mounted on a centos 
6.4 client, a solaris 5.8 client, and a solaris 5.6 client. The centos 
6.4 client and solaris 5.8 client are able to successfully execute an 
'ls *' on both the mounted shares, the solaris 5.6 client is not able to 
successfully execute 'ls *' on one share but is able to do so on the 
other. The two shares are each sub-directories of two partitions mounted 
on the centos 6.4 server. As far as I can tell, the two partitions are 
set up the same way, only difference is the naming. Other than the 
/etc/exports file, no other nfs-related configuration has been done on 
the server so the config files are the vanilla versions created when the 
packages are installed.

I would like to be able to have the solaris 5.6 client be able to mount 
a given share and operate in the same manner as the rest of the clients, 
execute 'ls *' and whatever else. My coworker who is the one that 
brought this to my attention suggested that the issue might be with nfs 
versions, since the old server used nfs3 and the new one can use nfs4, 
but I've explicitly forced nfs3 on all the mounts and it doesn't appear 
to make a difference.

Thanks!
Miranda


-----------------------------------------------------

oldnas1 (redhat 7.3 nfs server):
    oldnas1:/root/mroot 10# uname -a
       Linux oldnas1 2.4.27-CyberNAS_2.5.0.8 #2 SMP Fri Apr 15 21:00:27 
EDT 2005 i686 unknown
    oldnas1:/root/mroot 15# rpm -q -a|grep -i nfs
       nfs-utils-0.3.3-6.73
    oldnas1:/root/mroot 18# rpm -q -a | grep -i autofs
       autofs-3.1.7-28
    oldnas1:/root/mroot 20# rpcinfo -p
       program vers proto   port
        100000    2   tcp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  32768  status
        100024    1   tcp  32768  status
        100007    2   udp    869  ypbind
        100007    1   udp    869  ypbind
        100007    2   tcp    872  ypbind
        100007    1   tcp    872  ypbind
        100011    1   udp    627  rquotad
        100011    2   udp    627  rquotad
        100011    1   tcp    630  rquotad
        100011    2   tcp    630  rquotad
        100003    2   udp   2049  nfs
        100003    3   udp   2049  nfs
        100003    2   tcp   2049  nfs
        100003    3   tcp   2049  nfs
        100021    1   udp  32772  nlockmgr
        100021    3   udp  32772  nlockmgr
        100021    4   udp  32772  nlockmgr
        100021    1   tcp  32769  nlockmgr
        100021    3   tcp  32769  nlockmgr
        100021    4   tcp  32769  nlockmgr
        100005    1   udp  32773  mountd
        100005    1   tcp  32770  mountd
        100005    2   udp  32773  mountd
        100005    2   tcp  32770  mountd
        100005    3   udp  32773  mountd
        100005    3   tcp  32770  mountd
    oldnas1:/root/mroot 23# cat /etc/exports
       /aux1   xxx.xxx.xxx.0/24(rw,no_root_squash,insecure,async)
    oldnas1:/root/mroot 24# df -h
       Filesystem            Size  Used Avail Use% Mounted on
       /dev/hda2              20G  5.7G   13G  31% /
       /dev/hda1              45M  8.8M   34M  21% /boot
       none                 1010M     0 1010M   0% /dev/shm
       /dev/hda5              16G   33M   15G   1% /r2
       /dev/vm_aux1/vol0     920G  838G   35G  96% /aux1

boltzmann (centos 6.4 nfs server):
    boltzmann:/ 36# uname -a
       Linux boltzmann 2.6.32-358.11.1.el6.i686 #1 SMP Wed Jun 12 
01:01:27 UTC 2013 i686 i686 i386 GNU/Linux
    boltzmann:/ 37# rpm -q -a|grep -i nfs
       nfs-utils-1.2.3-36.el6.i686
       nfs4-acl-tools-0.3.3-6.el6.i686
       nfs-utils-lib-1.1.5-6.el6.i686
    boltzmann:/ 38# rpm -q -a | grep -i autofs
       autofs-5.0.5-74.el6_4.i686
       libsss_autofs-1.9.2-82.7.el6_4.i686
    boltzmann:/ 39# rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  39430  status
        100024    1   tcp  58291  status
        100007    2   udp   1017  ypbind
        100007    1   udp   1017  ypbind
        100007    2   tcp   1020  ypbind
        100007    1   tcp   1020  ypbind
        100021    1   udp  36412  nlockmgr
        100021    3   udp  36412  nlockmgr
        100021    4   udp  36412  nlockmgr
        100021    1   tcp  57032  nlockmgr
        100021    3   tcp  57032  nlockmgr
        100021    4   tcp  57032  nlockmgr
        100011    1   udp    875  rquotad
        100011    2   udp    875  rquotad
        100011    1   tcp    875  rquotad
        100011    2   tcp    875  rquotad
        100005    1   udp  49016  mountd
        100005    1   tcp  35130  mountd
        100005    2   udp  49472  mountd
        100005    2   tcp  55444  mountd
        100005    3   udp  55587  mountd
        100005    3   tcp  60747  mountd
        100003    2   tcp   2049  nfs
        100003    3   tcp   2049  nfs
        100003    4   tcp   2049  nfs
        100227    2   tcp   2049  nfs_acl
        100227    3   tcp   2049  nfs_acl
        100003    2   udp   2049  nfs
        100003    3   udp   2049  nfs
        100003    4   udp   2049  nfs
        100227    2   udp   2049  nfs_acl
        100227    3   udp   2049  nfs_acl
    boltzmann:/ 40# cat /etc/exports
       /scrs1_bolt xxx.xxx.xxx.0/24(rw,no_root_squash,insecure,async)
       /summit_bolt xxx.xxx.xxx.0/24(rw,no_root_squash,insecure,async)
    boltzmann:/ 41# df -h
       Filesystem            Size  Used Avail Use% Mounted on
       /dev/sdb2              50G   13G   37G  26% /
       tmpfs                 3.9G  2.1M  3.9G   1% /dev/shm
       /dev/sdb3             177G  188M  175G   1% /aux
       /dev/sda3             208G   44G  164G  21% /aux2

bigdog (solaris 5.6 nfs client):
    bigdog:/root/mroot 172#    uname -a
       SunOS bigdog 5.6 Generic_105181-39 sun4u sparc 
SUNW,UltraSPARC-IIi-cEngine
    bigdog:/root/mroot 174#    nfsstat -a

       Server nfs_acl:
       Version 2: (0 calls)
       null        getacl      setacl      getattr     access
       0 0%        0 0%        0 0%        0 0%        0 0%
       Version 3: (0 calls)
       null        getacl      setacl
       0 0%        0 0%        0 0%

       Client nfs_acl:
       Version 2: (1 calls)
       null        getacl      setacl      getattr     access
       0 0%        0 0%        0 0%        1 100%      0 0%
       Version 3: (1316 calls)
       null        getacl      setacl
       0 0%        1316 100%   0 0%
    bigdog:/root/mroot 176#    df -k
       Filesystem            kbytes    used   avail capacity  Mounted on
       /dev/dsk/c0t3d0s0      91570   59774   22639    73%    /
       /dev/dsk/c0t3d0s6     493354  392770   51249    89%    /usr
       /proc                      0       0       0     0%    /proc
       fd                         0       0       0     0%    /dev/fd
       /dev/dsk/c0t3d0s7     493354  416139   27880    94%    /aux
       /dev/dsk/c0t3d0s4    7422281 1474065 5873994    21%    /aux1
       swap                  192216       8  192208     1%    /tmp
       duke:/summit/home/projects/bigdog
                            2882715968 845114752 2008314512    30% 
/home/bigdog
bigdog:/root/mroot 200#    rpcinfo -p
    program vers proto   port  service
     100000    4   tcp    111  rpcbind
     100000    3   tcp    111  rpcbind
     100000    2   tcp    111  rpcbind
     100000    4   udp    111  rpcbind
     100000    3   udp    111  rpcbind
     100000    2   udp    111  rpcbind
     100007    3   udp  32774  ypbind
     100007    2   udp  32774  ypbind
     100007    1   udp  32774  ypbind
     100007    3   tcp  32771  ypbind
     100007    2   tcp  32771  ypbind
     100007    1   tcp  32771  ypbind
     100011    1   udp  32776  rquotad
     100002    2   udp  32777  rusersd
     100002    3   udp  32777  rusersd
     100002    2   tcp  32772  rusersd
     100002    3   tcp  32772  rusersd
     100012    1   udp  32778  sprayd
     100008    1   udp  32779  walld
     100001    2   udp  32780  rstatd
     100001    3   udp  32780  rstatd
     100001    4   udp  32780  rstatd
     100068    2   udp  32781
     100068    3   udp  32781
     100068    4   udp  32781
     100083    1   tcp  32773
     100221    1   tcp  32774
     100024    1   udp  32782  status
     100024    1   tcp  32775  status
     100021    1   udp   4045  nlockmgr
     100021    2   udp   4045  nlockmgr
     100021    3   udp   4045  nlockmgr
     100021    4   udp   4045  nlockmgr
     100021    1   tcp   4045  nlockmgr
     100021    2   tcp   4045  nlockmgr
     100021    3   tcp   4045  nlockmgr
     100021    4   tcp   4045  nlockmgr
     300598    1   udp  32786
     300598    1   tcp  32776
  805306368    1   udp  32786
  805306368    1   tcp  32776
  536870937    1   udp  41637
  536870937    1   tcp  32859

planck (solaris 5.8 nfs client):
    # uname -a
       SunOS planck 5.8 Generic_117350-38 sun4u sparc SUNW,Ultra-60
    # nfsstat -a

       Server nfs_acl:
       Version 2: (0 calls)
       null        getacl      setacl      getattr     access
       0 0%        0 0%        0 0%        0 0%        0 0%
       Version 3: (0 calls)
       null        getacl      setacl
       0 0%        0 0%        0 0%

       Client nfs_acl:
       Version 2: (1 calls)
       null        getacl      setacl      getattr     access
       0 0%        0 0%        0 0%        1 100%      0 0%
       Version 3: (270 calls)
       null        getacl      setacl
       0 0%        270 100%    0 0%
    # df -k
       Filesystem            kbytes    used   avail capacity  Mounted on
       /dev/dsk/c0t1d0s0    1018191  536202  420898    57%    /
       /dev/dsk/c0t1d0s6    2056211  843356 1151169    43%    /usr
       /proc                      0       0       0     0%    /proc
       fd                         0       0       0     0%    /dev/fd
       mnttab                     0       0       0     0% /etc/mnttab
       swap                 2456672      24 2456648     1% /var/run
       swap                 2456664      16 2456648     1%    /tmp
       /dev/dsk/c0t1d0s7    12244868 5677046 6445374    47%    /aux
       duke:/summit/home/projects/cshell
                            2882715968 845114748 2008314516    30% 
/home/cshell
# rpcinfo -p
    program vers proto   port  service
     100000    4   tcp    111  rpcbind
     100000    3   tcp    111  rpcbind
     100000    2   tcp    111  rpcbind
     100000    4   udp    111  rpcbind
     100000    3   udp    111  rpcbind
     100000    2   udp    111  rpcbind
     100007    3   udp  32774  ypbind
     100007    2   udp  32774  ypbind
     100007    1   udp  32774  ypbind
     100007    3   tcp  32771  ypbind
     100007    2   tcp  32771  ypbind
     100007    1   tcp  32771  ypbind
     100011    1   udp  32777  rquotad
     100002    2   udp  32778  rusersd
     100002    3   udp  32778  rusersd
     100002    2   tcp  32772  rusersd
     100002    3   tcp  32772  rusersd
     100024    1   udp  32779  status
     100024    1   tcp  32773  status
     100133    1   udp  32779
     100133    1   tcp  32773
     100012    1   udp  32780  sprayd
     100008    1   udp  32781  walld
     100001    2   udp  32782  rstatd
     100001    3   udp  32782  rstatd
     100001    4   udp  32782  rstatd
     100083    1   tcp  32774
     100221    1   tcp  32775
     100068    2   udp  32783
     100068    3   udp  32783
     100068    4   udp  32783
     100068    5   udp  32783
     300326    4   tcp  32776
     100021    1   udp   4045  nlockmgr
     100021    2   udp   4045  nlockmgr
     100021    3   udp   4045  nlockmgr
     100021    4   udp   4045  nlockmgr
     100021    1   tcp   4045  nlockmgr
     100021    2   tcp   4045  nlockmgr
     100021    3   tcp   4045  nlockmgr
     100021    4   tcp   4045  nlockmgr
     300598    1   udp  32798
     300598    1   tcp  32779
  805306368    1   udp  32798
  805306368    1   tcp  32779
     100249    1   udp  32799
     100249    1   tcp  32780

stefan (centos 6.4 nfs client):
    stefan:/root/mroot 21# uname -a
       Linux stefan 2.6.32-358.11.1.el6.i686 #1 SMP Wed Jun 12 01:01:27 
UTC 2013 i686 i686 i386 GNU/Linux
    stefan:/root/mroot 22# rpm -q -a|grep -i nfs
       nfs-utils-1.2.3-36.el6.i686
       nfs4-acl-tools-0.3.3-6.el6.i686
       nfs-utils-lib-1.1.5-6.el6.i686
    stefan:/root/mroot 23# rpm -q -a | grep -i autofs
       libsss_autofs-1.9.2-82.7.el6_4.i686
       autofs-5.0.5-74.el6_4.i686
    stefan:/root/mroot 24# rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  50165  status
        100024    1   tcp  38413  status
        100007    2   udp    719  ypbind
        100007    1   udp    719  ypbind
        100007    2   tcp    722  ypbind
        100007    1   tcp    722  ypbind
        100021    1   udp  44758  nlockmgr
        100021    3   udp  44758  nlockmgr
        100021    4   udp  44758  nlockmgr
        100021    1   tcp  59666  nlockmgr
        100021    3   tcp  59666  nlockmgr
        100021    4   tcp  59666  nlockmgr
    stefan:/root/mroot 25# df -h
       Filesystem            Size  Used Avail Use% Mounted on
       /dev/sda2              40G  9.1G   29G  25% /
       tmpfs                 3.9G  8.8M  3.9G   1% /dev/shm
       /dev/sda4             372G  195M  353G   1% /aux
       /dev/sda3              40G  176M   38G   1% /r3
       duke:/summit/home/projects/bigdog
                             2.7T  806G  1.9T  30% /home/bigdog
       irtfweb:/htdocs       459G  358G   96G  79% /netdisks/htdocs
       duke:/scrs1           2.7T   70G  2.6T   3% /netdisks/scrs1

-----------------------------------------------------
various hosts' output:

boltzmann:/ 42# exportfs -auv
boltzmann:/ 43# exportfs -rv
exporting xxx.xxx.xxx.0/24:/aux2/summit_bolt
exporting xxx.xxx.xxx.0/24:/aux/scrs1_bolt
boltzmann:/ 50#  ls -la /scrs1_bolt/ /summit_bolt/
/scrs1_bolt/:
total 12
drwxrwxrwx  3 root   bin     4096 Jul  3 12:35 ./
drwxr-xr-x. 4 root   root    4096 Jul  3 09:32 ../
-rw-r--r--  1 root   bin        0 Jul  3 09:26 boltzmann_test_scrs1
drwxrwxr-x  3 bigdog project 4096 Jul  3 12:35 dir1/
-rw-r--r--  1 root   bin        0 Jul  3 12:31 SCRS1_BOLT

/summit_bolt/:
total 8
drwxrwxrwx 2 root bin  4096 Jul  3 12:53 ./
drwxr-xr-x 7 root root 4096 Jul  3 09:32 ../
-rw-r--r-- 1 root bin     0 Jul  3 09:26 boltzmann_test_summit
-rw-r--r-- 1 root bin     0 Jul  3 12:32 SUMMIT_BOLT

stefan:/root/mroot 26# mkdir /mnt/b1 /mnt/b2
stefan:/root/mroot 29# mount -t nfs -o nfsvers=3 boltzmann:/scrs1_bolt 
/mnt/b1
stefan:/root/mroot 30# mount -t nfs -o nfsvers=3 boltzmann:/summit_bolt 
/mnt/b2
stefan:/root/mroot 31# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              40G  9.1G   29G  25% /
tmpfs                 3.9G  8.8M  3.9G   1% /dev/shm
/dev/sda4             372G  195M  353G   1% /aux
/dev/sda3              40G  176M   38G   1% /r3
duke:/summit/home/projects/bigdog
                       2.7T  806G  1.9T  30% /home/bigdog
irtfweb:/htdocs       459G  358G   96G  79% /netdisks/htdocs
duke:/scrs1           2.7T   70G  2.6T   3% /netdisks/scrs1
boltzmann:/scrs1_bolt
                       177G  188M  175G   1% /mnt/b1
boltzmann:/summit_bolt
                       208G   44G  164G  21% /mnt/b2
stefan:/root/mroot 32# ls -la /mnt/b1/*
-rw-r--r-- 1 root   bin        0 Jul  3 09:26 /mnt/b1/boltzmann_test_scrs1
-rw-r--r-- 1 root   bin        0 Jul  3 12:31 /mnt/b1/SCRS1_BOLT

/mnt/b1/dir1:
total 12
drwxrwxr-x 3 bigdog project 4096 Jul  3 12:35 ./
drwxrwxrwx 3 root   bin     4096 Jul  3 12:35 ../
drwxrwxr-x 2 bigdog project 4096 Jul  3 12:37 dir2/
stefan:/root/mroot 33# ls -la /mnt/b2/*
-rw-r--r-- 1 root bin 0 Jul  3 09:26 /mnt/b2/boltzmann_test_summit
-rw-r--r-- 1 root bin 0 Jul  3 12:32 /mnt/b2/SUMMIT_BOLT

bigdog:/root/mroot 182#    mkdir /mnt/b1 /mnt/b2
bigdog:/root/mroot 183#    mount -F nfs -o nfsvers=3 
boltzmann:/scrs1_bolt /mnt/b1
bigdog:/root/mroot 184#    mount -F nfs -o nfsvers=3 
boltzmann:/summit_bolt /mnt/b2
bigdog:/root/mroot 185#    df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t3d0s0      91570   59775   22638    73%    /
/dev/dsk/c0t3d0s6     493354  392770   51249    89%    /usr
/proc                      0       0       0     0%    /proc
fd                         0       0       0     0%    /dev/fd
/dev/dsk/c0t3d0s7     493354  416139   27880    94%    /aux
/dev/dsk/c0t3d0s4    7422281 1474065 5873994    21%    /aux1
swap                  192176       8  192168     1%    /tmp
duke:/summit/home/projects/bigdog
                      2882715968 845116312 2008312952    30% /home/bigdog
boltzmann:/scrs1_bolt
                      184615952  191824 182548536     1%    /mnt/b1
boltzmann:/summit_bolt
                      217642344 45699464 171942880    21%    /mnt/b2
bigdog:/root/mroot 186#    ls -la /mnt/b1/*
ls: No match.
bigdog:/root/mroot 187#    ls -la /mnt/b2/*
-rw-r--r--   1 root     other          0 Jul  3 12:32 /mnt/b2/SUMMIT_BOLT
-rw-r--r--   1 root     other          0 Jul  3 12:53 
/mnt/b2/bigdog_mounted_nfs3
-rw-r--r--   1 root     other          0 Jul  3 09:26 
/mnt/b2/boltzmann_test_summit

planck:
# mkdir /mnt/b1 /mnt/b2
# mount -F nfs -o vers=3 boltzmann:/scrs1_bolt /mnt/b1
# mount -F nfs -o vers=3 boltzmann:/summit_bolt /mnt/b2
# df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t1d0s0    1018191  536204  420896    57%    /
/dev/dsk/c0t1d0s6    2056211  843356 1151169    43%    /usr
/proc                      0       0       0     0%    /proc
fd                         0       0       0     0%    /dev/fd
mnttab                     0       0       0     0%    /etc/mnttab
swap                 2456600      24 2456576     1%    /var/run
swap                 2456592      16 2456576     1%    /tmp
/dev/dsk/c0t1d0s7    12244868 5677046 6445374    47%    /aux
duke:/summit/home/projects/cshell
                      2882715968 845116316 2008312948    30% /home/cshell
boltzmann:/scrs1_bolt
                      184615956  191828 182548540     1%    /mnt/b1
boltzmann:/summit_bolt
                      217642344 45699460 171942884    21%    /mnt/b2
# ls -la /mnt/b1/*
-rw-r--r--   1 root     other          0 Jul  3 12:31 /mnt/b1/SCRS1_BOLT
-rw-r--r--   1 root     other          0 Jul  3 09:26 
/mnt/b1/boltzmann_test_scrs1

/mnt/b1/dir1:
total 24
drwxrwxr-x   3 bigdog   project     4096 Jul  3 12:35 .
drwxrwxrwx   3 root     other       4096 Jul  3 12:35 ..
drwxrwxr-x   2 bigdog   project     4096 Jul  3 12:37 dir2
# ls -la /mnt/b2/*
-rw-r--r--   1 root     other          0 Jul  3 12:32 /mnt/b2/SUMMIT_BOLT
-rw-r--r--   1 root     other          0 Jul  3 09:26 
/mnt/b2/boltzmann_test_summit

boltzmann:/ 45# netstat -an | grep boltzmann:2049
tcp        0      0 boltzmann:2049          stefan:776 ESTABLISHED
tcp        0      0 boltzmann:2049          planck:686 ESTABLISHED
tcp        0      0 boltzmann:2049          bigdog:960 ESTABLISHED