[CentOS] Windows 2008R2 AD, kerberos, NFSv4

Mon Apr 23 22:18:07 UTC 2012
James A. Peltier <jpeltier at sfu.ca>

Please provide your smb.conf and krb5.conf files as well.  BTW: the createupn is not required on Win2K8R2 as this credential is passed now (according to MS)

----- Original Message -----
| Hi,
| 
| I'm trying to set up NFSv4 on two boxes (centos 5.5)  and have it
| authenticate against our Windows 2008R2 AD server acting as the KDC.
|  (samba/winbind is running ok with "idmap config MYCOMPANY: backend =
|  rid"
| so we have identical ids across the servers.)
| 
| I can mount my test directory fine via NFSv4 *without* the sec=krb5
| option.
|  However, once I put the sec=krb5 option in, then I get a mount
|  error:
|  "mount.nfs4: Permission denied" and rpc.gssd reports: "Failed to
|  obtain
| machine credentials for connection to server"
| 
| The computers have an AD computer account and for the
| service-principal, I
| created an AD user account "nfsHostname" and mapped the UPN e.g. NFS/
| hostname.mycompany.tv at MYCOMPANY.TV to it using ktpass.
| 
| This is the closest post similar to my issue I could find:
| http://lists.centos.org/pipermail/centos/2010-July/096378.html
|    However,
| I'm trying not to run the createupn command via smbutils.
| Side note:
| Eventually we will also be using a HDS nas which doesn't provide us
| with
| samba net utils (e.g. net ads join createupn) only their proprietary
| webadmin/cli.  When that nas joined our AD domain, it created a
| computer
| account with SPNs of HOST/HOSTNAME, HOST/hostname.MYCOMPANY.TV and a
| UPN of
| HOST/hostname.mycompany.tv at MYCOMPANY.TV  And the HDS nas only wants
| encryption type:  des-cbc-crc:normal.  This is why on my test nfs
| server
| (nas002), I'm trying to use the same limited commands as I would if I
| were
| using the HDS nas.
| 
| Any suggestions where to look next or get more verbose info from
| kerberos/KDC or the nfs server?  (nothing shows up in either syslog
| --
| plus, I'm not all that familiar with kerberos.)
| 
| thanks in advance!
| JA.
| 
| 
| 
| info:
| 10.100.1.11  KDC server (Windows 2008 R2, AD)
| 10.100.1.35  bk001  (nfsv4 client, kernel 2.6.18-194.32.1.el5)
| 10.100.1.82  nas002 (nfsv4 server, kernel 2.6.18-194.32.1.el5)
| 10.100.1.99  monitoring server
| 
| intsalled on both nfsv4 client and server:
| nfs-utils.x86_64 1.0.9-60.el5
| nfs-utils-lib.x86_64 1.0.8-7.9.el5
| nfs4-acl-tools.x86_64 0.3.3-3.el5
| krb5-workstation.x86_64 1.6.1-70.el5
| samba (nas002)  3.3.8-0.52.el5_5.2
| samba (bk001)   3.5.10-0.107.el5
| 
| 
| 
| [root at bk001 ~]# net ads testjoin
| Join is OK
| 
| [root at bk001 ~]# kinit administrator at MYCOMPANY.TV
| Password for administrator at MYCOMPANY.TV:
| 
| [root at bk001 ~]# kinit nfs/nas002.mycompany.tv at MYCOMPANY.TV
| Password for nfs/nas002.mycompany.tv at MYCOMPANY.TV:
| 
| [root at bk001 ~]# klist
| Ticket cache: FILE:/tmp/krb5cc_0
| Default principal: nfs/nas002.mycompany.tv at MYCOMPANY.TV
| 
| Valid starting     Expires            Service principal
| 04/13/12 16:08:51  04/14/12 02:08:51
|  krbtgt/MYCOMPANY.TV at MYCOMPANY.TV
|         renew until 04/16/12 16:08:51
| 
| 
| Kerberos 4 ticket cache: /tmp/tkt0
| klist: You have no tickets cached
| 
| 
| [root at bk001 ~]# showmount -e nas002.mycompany.tv
| Export list for nas002.mycompany.tv:
| /array gss/krb5,*
| 
| 
| [root at bk001 ~]# mount -v -t nfs4 -o proto=tcp,sec=krb5
| nas002.mycompany.tv:/
| /mnt/nfs4test
| Warning: rpc.idmapd appears not to be running.
|          All uids will be mapped to the nobody uid.
| Warning: rpc.gssd appears not to be running.
| mount: pinging: prog 100003 vers 4 prot tcp port 2049
| mount.nfs4: Permission denied
| 
| [root at bk001 ~]# ps -elf | egrep 'gss|idmap'
| 1 S root      2498     1  0  75   0 -  8016 -      Apr12 ?
|        00:00:00
| rpc.gssd -rrrvvvv
| 1 S root      4575     1  0  76   0 - 14833 -      Apr12 ?
|        00:00:00
| rpc.idmapd -vvv
| 
| 
| [root at bk001 ~]# tail /var/log/messages
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]: New client: 16
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: handling krb5 upcall
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]: New client: 17
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Opened
| /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: Using keytab file
| '/etc/krb5.keytab'
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: WARNING: Failed to obtain
| machine
| credentials for connection to server nas002.mycompany.tv
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: doing error downcall
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Stale client: 16
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]:  -> closed
| /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Stale client: 17
| Apr 13 16:09:09 bk001 rpc.idmapd[4575]:  -> closed
| /var/lib/nfs/rpc_pipefs/nfs/clnt17/idmap
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: destroying client clnt17
| Apr 13 16:09:09 bk001 rpc.gssd[2498]: destroying client clnt16
| 
| 
| 
| tshark capture of commands I performed (above):
| [root at bk001 ~]# cat /var/tmp/tshark_041312-1608.out
| 366   9.948504  10.100.1.35 -> 10.100.1.11  TCP 42564 > kerberos
| [SYN]
| Seq=0 Win=5840 Len=0 MSS=1460 TSV=86719599 TSER=0 WS=7
| 367   9.948813  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42564
| [SYN, ACK]
| Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396813568 TSER=86719599
| 368   9.948824  10.100.1.35 -> 10.100.1.11  TCP 42564 > kerberos
| [ACK]
| Seq=1 Ack=1 Win=5840 Len=0 TSV=86719599 TSER=396813568
| 369   9.948849  10.100.1.35 -> 10.100.1.11  KRB5 AS-REQ
| 370   9.949976  10.100.1.11 -> 10.100.1.35  KRB5 KRB Error:
| KRB5KDC_ERR_PREAUTH_REQUIRED
| 371   9.949982  10.100.1.35 -> 10.100.1.11  TCP 42564 > kerberos
| [ACK]
| Seq=181 Ack=154 Win=6432 Len=0 TSV=86719600 TSER=396813568
| 372   9.950031  10.100.1.35 -> 10.100.1.11  TCP 42564 > kerberos
| [FIN, ACK]
| Seq=181 Ack=154 Win=6432 Len=0 TSV=86719600 TSER=396813568
| 373   9.950288  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42564
| [ACK]
| Seq=154 Ack=182 Win=65160 Len=0 TSV=396813568 TSER=86719600
| 374   9.950297  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42564
| [RST, ACK]
| Seq=154 Ack=182 Win=0 Len=0
| 444  11.840921  10.100.1.35 -> 10.100.1.11  TCP 42565 > kerberos
| [SYN]
| Seq=0 Win=5840 Len=0 MSS=1460 TSV=86721491 TSER=0 WS=7
| 446  11.841178  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42565
| [SYN, ACK]
| Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396813757 TSER=86721491
| 447  11.841185  10.100.1.35 -> 10.100.1.11  TCP 42565 > kerberos
| [ACK]
| Seq=1 Ack=1 Win=5840 Len=0 TSV=86721491 TSER=396813757
| 448  11.841206  10.100.1.35 -> 10.100.1.11  KRB5 AS-REQ
| 449  11.842812  10.100.1.11 -> 10.100.1.35  TCP [TCP segment of a
| reassembled PDU]
| 450  11.842817  10.100.1.35 -> 10.100.1.11  TCP 42565 > kerberos
| [ACK]
| Seq=259 Ack=1449 Win=8688 Len=0 TSV=86721493 TSER=396813757
| 451  11.842819  10.100.1.11 -> 10.100.1.35  KRB5 AS-REP
| 452  11.842822  10.100.1.35 -> 10.100.1.11  TCP 42565 > kerberos
| [ACK]
| Seq=259 Ack=1518 Win=8688 Len=0 TSV=86721493 TSER=396813757
| 453  11.842852  10.100.1.35 -> 10.100.1.11  TCP 42565 > kerberos
| [FIN, ACK]
| Seq=259 Ack=1518 Win=8688 Len=0 TSV=86721493 TSER=396813757
| 454  11.843043  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42565
| [ACK]
| Seq=1518 Ack=260 Win=65160 Len=0 TSV=396813758 TSER=86721493
| 455  11.843050  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42565
| [RST, ACK]
| Seq=1518 Ack=260 Win=0 Len=0
| 827  21.821693  10.100.1.35 -> 10.100.1.11  TCP 42566 > kerberos
| [SYN]
| Seq=0 Win=5840 Len=0 MSS=1460 TSV=86731472 TSER=0 WS=7
| 828  21.821920  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42566
| [SYN, ACK]
| Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396814755 TSER=86731472
| 829  21.821930  10.100.1.35 -> 10.100.1.11  TCP 42566 > kerberos
| [ACK]
| Seq=1 Ack=1 Win=5840 Len=0 TSV=86731472 TSER=396814755
| 830  21.821958  10.100.1.35 -> 10.100.1.11  KRB5 AS-REQ
| 831  21.822968  10.100.1.11 -> 10.100.1.35  KRB5 AS-REP
| 832  21.822974  10.100.1.35 -> 10.100.1.11  TCP 42566 > kerberos
| [ACK]
| Seq=191 Ack=618 Win=6787 Len=0 TSV=86731473 TSER=396814756
| 833  21.823003  10.100.1.35 -> 10.100.1.11  TCP 42566 > kerberos
| [FIN, ACK]
| Seq=191 Ack=618 Win=6787 Len=0 TSV=86731473 TSER=396814756
| 835  21.823278  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42566
| [ACK]
| Seq=618 Ack=192 Win=65160 Len=0 TSV=396814756 TSER=86731473
| 836  21.823287  10.100.1.11 -> 10.100.1.35  TCP kerberos > 42566
| [RST, ACK]
| Seq=618 Ack=192 Win=0 Len=0
| 1472  39.980317  10.100.1.35 -> 10.100.1.82  TCP 40520 > nfs [SYN]
| Seq=0
| Win=5840 Len=0 MSS=1460 TSV=86749629 TSER=0 WS=7
| 1473  39.980491  10.100.1.82 -> 10.100.1.35  TCP nfs > 40520 [SYN,
| ACK]
| Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=3789493491 TSER=86749629 WS=7
| 1474  39.980498  10.100.1.35 -> 10.100.1.82  TCP 40520 > nfs [ACK]
| Seq=1
| Ack=1 Win=5888 Len=0 TSV=86749631 TSER=3789493491
| 1475  39.980533  10.100.1.35 -> 10.100.1.82  NFS V4 NULL Call
| 1476  39.980701  10.100.1.82 -> 10.100.1.35  TCP nfs > 40520 [ACK]
| Seq=1
| Ack=45 Win=5888 Len=0 TSV=3789493492 TSER=86749631
| 1477  39.980705  10.100.1.82 -> 10.100.1.35  NFS V4 NULL Reply (Call
| In
| 1475)
| 1478  39.980707  10.100.1.35 -> 10.100.1.82  TCP 40520 > nfs [ACK]
| Seq=45
| Ack=29 Win=5888 Len=0 TSV=86749631 TSER=3789493492
| 1479  39.980733  10.100.1.35 -> 10.100.1.82  TCP 40520 > nfs [FIN,
| ACK]
| Seq=45 Ack=29 Win=5888 Len=0 TSV=86749631 TSER=3789493492
| 1480  39.980896  10.100.1.82 -> 10.100.1.35  TCP nfs > 40520 [FIN,
| ACK]
| Seq=29 Ack=46 Win=5888 Len=0 TSV=3789493492 TSER=86749631
| 1481  39.980901  10.100.1.35 -> 10.100.1.82  TCP 40520 > nfs [ACK]
| Seq=46
| Ack=30 Win=5888 Len=0 TSV=86749631 TSER=3789493492
| 1482  40.001039  10.100.1.35 -> 10.100.1.82  TCP connendp > nfs [SYN]
| Seq=0
| Win=5840 Len=0 MSS=1460 TSV=86749651 TSER=0 WS=7
| 1483  40.001210  10.100.1.82 -> 10.100.1.35  TCP nfs > connendp [SYN,
| ACK]
| Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=3789493512 TSER=86749651 WS=7
| 1484  40.001221  10.100.1.35 -> 10.100.1.82  TCP connendp > nfs [ACK]
| Seq=1
| Ack=1 Win=5888 Len=0 TSV=86749651 TSER=3789493512
| 1485  40.001244  10.100.1.35 -> 10.100.1.82  NFS V4 NULL Call
| 1486  40.001409  10.100.1.82 -> 10.100.1.35  TCP nfs > connendp [ACK]
| Seq=1
| Ack=45 Win=5888 Len=0 TSV=3789493512 TSER=86749651
| 1487  40.001414  10.100.1.82 -> 10.100.1.35  NFS V4 NULL Reply (Call
| In
| 1485)
| 1488  40.001418  10.100.1.35 -> 10.100.1.82  TCP connendp > nfs [ACK]
| Seq=45 Ack=29 Win=5888 Len=0 TSV=86749652 TSER=3789493512
| 1489  40.002363  10.100.1.35 -> 10.100.1.82  TCP connendp > nfs [FIN,
| ACK]
| Seq=45 Ack=29 Win=5888 Len=0 TSV=86749653 TSER=3789493512
| 1490  40.002526  10.100.1.82 -> 10.100.1.35  TCP nfs > connendp [FIN,
| ACK]
| Seq=29 Ack=46 Win=5888 Len=0 TSV=3789493513 TSER=86749653
| 1491  40.002532  10.100.1.35 -> 10.100.1.82  TCP connendp > nfs [ACK]
| Seq=46 Ack=30 Win=5888 Len=0 TSV=86749653 TSER=3789493513
| 1493  40.002880  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: New client: 16\n
| 1497  40.003611  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: handling krb5 upcall \n
| 1498  40.004069  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: New client: 17\n
| 1499  40.004489  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: Opened /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap\n
| 1500  40.004949  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: Using keytab file '/etc/krb5.keytab' \n
| 1501  40.005369  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: WARNING: Failed to obtain machine credentials for
| connection to server nas002.mycompany.tv \n
| 1502  40.005829  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: doing error downcall \n
| 1503  40.012862  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: Stale client: 16\n
| 1504  40.013326  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: \t-> closed
| /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap\n
| 1505  40.013740  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: Stale client: 17\n
| 1506  40.014157  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.WARNING:
| rpc.idmapd[4575]: \t-> closed
| /var/lib/nfs/rpc_pipefs/nfs/clnt17/idmap\n
| 1507  40.014621  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: destroying client clnt17 \n
| 1508  40.015082  10.100.1.35 -> 10.100.1.99  Syslog DAEMON.ERR:
| rpc.gssd[2498]: destroying client clnt16 \n
| [root at bk001 ~]#
| _______________________________________________
| CentOS mailing list
| CentOS at centos.org
| http://lists.centos.org/mailman/listinfo/centos
| 

-- 
James A. Peltier
Manager, IT Services - Research Computing Group
Simon Fraser University - Burnaby Campus
Phone   : 778-782-6573
Fax     : 778-782-3045
E-Mail  : jpeltier at sfu.ca
Website : http://www.sfu.ca/itservices
          http://blogs.sfu.ca/people/jpeltier

Success is to be measured not so much by the position that one has reached
in life but as by the obstacles they have overcome. - Booker T. Washington