Hi,
I'm trying to set up NFSv4 on two boxes (centos 5.5) and have it authenticate against our Windows 2008R2 AD server acting as the KDC. (samba/winbind is running ok with "idmap config MYCOMPANY: backend = rid" so we have identical ids across the servers.)
I can mount my test directory fine via NFSv4 *without* the sec=krb5 option. However, once I put the sec=krb5 option in, then I get a mount error: "mount.nfs4: Permission denied" and rpc.gssd reports: "Failed to obtain machine credentials for connection to server"
The computers have an AD computer account and for the service-principal, I created an AD user account "nfsHostname" and mapped the UPN e.g. NFS/ hostname.mycompany.tv@MYCOMPANY.TV to it using ktpass.
This is the closest post similar to my issue I could find: http://lists.centos.org/pipermail/centos/2010-July/096378.html However, I'm trying not to run the createupn command via smbutils. Side note: Eventually we will also be using a HDS nas which doesn't provide us with samba net utils (e.g. net ads join createupn) only their proprietary webadmin/cli. When that nas joined our AD domain, it created a computer account with SPNs of HOST/HOSTNAME, HOST/hostname.MYCOMPANY.TV and a UPN of HOST/hostname.mycompany.tv@MYCOMPANY.TV And the HDS nas only wants encryption type: des-cbc-crc:normal. This is why on my test nfs server (nas002), I'm trying to use the same limited commands as I would if I were using the HDS nas.
Any suggestions where to look next or get more verbose info from kerberos/KDC or the nfs server? (nothing shows up in either syslog -- plus, I'm not all that familiar with kerberos.)
thanks in advance! JA.
info: 10.100.1.11 KDC server (Windows 2008 R2, AD) 10.100.1.35 bk001 (nfsv4 client, kernel 2.6.18-194.32.1.el5) 10.100.1.82 nas002 (nfsv4 server, kernel 2.6.18-194.32.1.el5) 10.100.1.99 monitoring server
intsalled on both nfsv4 client and server: nfs-utils.x86_64 1.0.9-60.el5 nfs-utils-lib.x86_64 1.0.8-7.9.el5 nfs4-acl-tools.x86_64 0.3.3-3.el5 krb5-workstation.x86_64 1.6.1-70.el5 samba (nas002) 3.3.8-0.52.el5_5.2 samba (bk001) 3.5.10-0.107.el5
[root@bk001 ~]# net ads testjoin Join is OK
[root@bk001 ~]# kinit administrator@MYCOMPANY.TV Password for administrator@MYCOMPANY.TV:
[root@bk001 ~]# kinit nfs/nas002.mycompany.tv@MYCOMPANY.TV Password for nfs/nas002.mycompany.tv@MYCOMPANY.TV:
[root@bk001 ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: nfs/nas002.mycompany.tv@MYCOMPANY.TV
Valid starting Expires Service principal 04/13/12 16:08:51 04/14/12 02:08:51 krbtgt/MYCOMPANY.TV@MYCOMPANY.TV renew until 04/16/12 16:08:51
Kerberos 4 ticket cache: /tmp/tkt0 klist: You have no tickets cached
[root@bk001 ~]# showmount -e nas002.mycompany.tv Export list for nas002.mycompany.tv: /array gss/krb5,*
[root@bk001 ~]# mount -v -t nfs4 -o proto=tcp,sec=krb5 nas002.mycompany.tv:/ /mnt/nfs4test Warning: rpc.idmapd appears not to be running. All uids will be mapped to the nobody uid. Warning: rpc.gssd appears not to be running. mount: pinging: prog 100003 vers 4 prot tcp port 2049 mount.nfs4: Permission denied
[root@bk001 ~]# ps -elf | egrep 'gss|idmap' 1 S root 2498 1 0 75 0 - 8016 - Apr12 ? 00:00:00 rpc.gssd -rrrvvvv 1 S root 4575 1 0 76 0 - 14833 - Apr12 ? 00:00:00 rpc.idmapd -vvv
[root@bk001 ~]# tail /var/log/messages Apr 13 16:09:09 bk001 rpc.idmapd[4575]: New client: 16 Apr 13 16:09:09 bk001 rpc.gssd[2498]: handling krb5 upcall Apr 13 16:09:09 bk001 rpc.idmapd[4575]: New client: 17 Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Opened /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap Apr 13 16:09:09 bk001 rpc.gssd[2498]: Using keytab file '/etc/krb5.keytab' Apr 13 16:09:09 bk001 rpc.gssd[2498]: WARNING: Failed to obtain machine credentials for connection to server nas002.mycompany.tv Apr 13 16:09:09 bk001 rpc.gssd[2498]: doing error downcall Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Stale client: 16 Apr 13 16:09:09 bk001 rpc.idmapd[4575]: -> closed /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap Apr 13 16:09:09 bk001 rpc.idmapd[4575]: Stale client: 17 Apr 13 16:09:09 bk001 rpc.idmapd[4575]: -> closed /var/lib/nfs/rpc_pipefs/nfs/clnt17/idmap Apr 13 16:09:09 bk001 rpc.gssd[2498]: destroying client clnt17 Apr 13 16:09:09 bk001 rpc.gssd[2498]: destroying client clnt16
tshark capture of commands I performed (above): [root@bk001 ~]# cat /var/tmp/tshark_041312-1608.out 366 9.948504 10.100.1.35 -> 10.100.1.11 TCP 42564 > kerberos [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=86719599 TSER=0 WS=7 367 9.948813 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42564 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396813568 TSER=86719599 368 9.948824 10.100.1.35 -> 10.100.1.11 TCP 42564 > kerberos [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=86719599 TSER=396813568 369 9.948849 10.100.1.35 -> 10.100.1.11 KRB5 AS-REQ 370 9.949976 10.100.1.11 -> 10.100.1.35 KRB5 KRB Error: KRB5KDC_ERR_PREAUTH_REQUIRED 371 9.949982 10.100.1.35 -> 10.100.1.11 TCP 42564 > kerberos [ACK] Seq=181 Ack=154 Win=6432 Len=0 TSV=86719600 TSER=396813568 372 9.950031 10.100.1.35 -> 10.100.1.11 TCP 42564 > kerberos [FIN, ACK] Seq=181 Ack=154 Win=6432 Len=0 TSV=86719600 TSER=396813568 373 9.950288 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42564 [ACK] Seq=154 Ack=182 Win=65160 Len=0 TSV=396813568 TSER=86719600 374 9.950297 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42564 [RST, ACK] Seq=154 Ack=182 Win=0 Len=0 444 11.840921 10.100.1.35 -> 10.100.1.11 TCP 42565 > kerberos [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=86721491 TSER=0 WS=7 446 11.841178 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42565 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396813757 TSER=86721491 447 11.841185 10.100.1.35 -> 10.100.1.11 TCP 42565 > kerberos [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=86721491 TSER=396813757 448 11.841206 10.100.1.35 -> 10.100.1.11 KRB5 AS-REQ 449 11.842812 10.100.1.11 -> 10.100.1.35 TCP [TCP segment of a reassembled PDU] 450 11.842817 10.100.1.35 -> 10.100.1.11 TCP 42565 > kerberos [ACK] Seq=259 Ack=1449 Win=8688 Len=0 TSV=86721493 TSER=396813757 451 11.842819 10.100.1.11 -> 10.100.1.35 KRB5 AS-REP 452 11.842822 10.100.1.35 -> 10.100.1.11 TCP 42565 > kerberos [ACK] Seq=259 Ack=1518 Win=8688 Len=0 TSV=86721493 TSER=396813757 453 11.842852 10.100.1.35 -> 10.100.1.11 TCP 42565 > kerberos [FIN, ACK] Seq=259 Ack=1518 Win=8688 Len=0 TSV=86721493 TSER=396813757 454 11.843043 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42565 [ACK] Seq=1518 Ack=260 Win=65160 Len=0 TSV=396813758 TSER=86721493 455 11.843050 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42565 [RST, ACK] Seq=1518 Ack=260 Win=0 Len=0 827 21.821693 10.100.1.35 -> 10.100.1.11 TCP 42566 > kerberos [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=86731472 TSER=0 WS=7 828 21.821920 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42566 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 TSV=396814755 TSER=86731472 829 21.821930 10.100.1.35 -> 10.100.1.11 TCP 42566 > kerberos [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=86731472 TSER=396814755 830 21.821958 10.100.1.35 -> 10.100.1.11 KRB5 AS-REQ 831 21.822968 10.100.1.11 -> 10.100.1.35 KRB5 AS-REP 832 21.822974 10.100.1.35 -> 10.100.1.11 TCP 42566 > kerberos [ACK] Seq=191 Ack=618 Win=6787 Len=0 TSV=86731473 TSER=396814756 833 21.823003 10.100.1.35 -> 10.100.1.11 TCP 42566 > kerberos [FIN, ACK] Seq=191 Ack=618 Win=6787 Len=0 TSV=86731473 TSER=396814756 835 21.823278 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42566 [ACK] Seq=618 Ack=192 Win=65160 Len=0 TSV=396814756 TSER=86731473 836 21.823287 10.100.1.11 -> 10.100.1.35 TCP kerberos > 42566 [RST, ACK] Seq=618 Ack=192 Win=0 Len=0 1472 39.980317 10.100.1.35 -> 10.100.1.82 TCP 40520 > nfs [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=86749629 TSER=0 WS=7 1473 39.980491 10.100.1.82 -> 10.100.1.35 TCP nfs > 40520 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=3789493491 TSER=86749629 WS=7 1474 39.980498 10.100.1.35 -> 10.100.1.82 TCP 40520 > nfs [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=86749631 TSER=3789493491 1475 39.980533 10.100.1.35 -> 10.100.1.82 NFS V4 NULL Call 1476 39.980701 10.100.1.82 -> 10.100.1.35 TCP nfs > 40520 [ACK] Seq=1 Ack=45 Win=5888 Len=0 TSV=3789493492 TSER=86749631 1477 39.980705 10.100.1.82 -> 10.100.1.35 NFS V4 NULL Reply (Call In 1475) 1478 39.980707 10.100.1.35 -> 10.100.1.82 TCP 40520 > nfs [ACK] Seq=45 Ack=29 Win=5888 Len=0 TSV=86749631 TSER=3789493492 1479 39.980733 10.100.1.35 -> 10.100.1.82 TCP 40520 > nfs [FIN, ACK] Seq=45 Ack=29 Win=5888 Len=0 TSV=86749631 TSER=3789493492 1480 39.980896 10.100.1.82 -> 10.100.1.35 TCP nfs > 40520 [FIN, ACK] Seq=29 Ack=46 Win=5888 Len=0 TSV=3789493492 TSER=86749631 1481 39.980901 10.100.1.35 -> 10.100.1.82 TCP 40520 > nfs [ACK] Seq=46 Ack=30 Win=5888 Len=0 TSV=86749631 TSER=3789493492 1482 40.001039 10.100.1.35 -> 10.100.1.82 TCP connendp > nfs [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=86749651 TSER=0 WS=7 1483 40.001210 10.100.1.82 -> 10.100.1.35 TCP nfs > connendp [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=3789493512 TSER=86749651 WS=7 1484 40.001221 10.100.1.35 -> 10.100.1.82 TCP connendp > nfs [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=86749651 TSER=3789493512 1485 40.001244 10.100.1.35 -> 10.100.1.82 NFS V4 NULL Call 1486 40.001409 10.100.1.82 -> 10.100.1.35 TCP nfs > connendp [ACK] Seq=1 Ack=45 Win=5888 Len=0 TSV=3789493512 TSER=86749651 1487 40.001414 10.100.1.82 -> 10.100.1.35 NFS V4 NULL Reply (Call In 1485) 1488 40.001418 10.100.1.35 -> 10.100.1.82 TCP connendp > nfs [ACK] Seq=45 Ack=29 Win=5888 Len=0 TSV=86749652 TSER=3789493512 1489 40.002363 10.100.1.35 -> 10.100.1.82 TCP connendp > nfs [FIN, ACK] Seq=45 Ack=29 Win=5888 Len=0 TSV=86749653 TSER=3789493512 1490 40.002526 10.100.1.82 -> 10.100.1.35 TCP nfs > connendp [FIN, ACK] Seq=29 Ack=46 Win=5888 Len=0 TSV=3789493513 TSER=86749653 1491 40.002532 10.100.1.35 -> 10.100.1.82 TCP connendp > nfs [ACK] Seq=46 Ack=30 Win=5888 Len=0 TSV=86749653 TSER=3789493513 1493 40.002880 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: New client: 16\n 1497 40.003611 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: handling krb5 upcall \n 1498 40.004069 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: New client: 17\n 1499 40.004489 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: Opened /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap\n 1500 40.004949 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: Using keytab file '/etc/krb5.keytab' \n 1501 40.005369 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: WARNING: Failed to obtain machine credentials for connection to server nas002.mycompany.tv \n 1502 40.005829 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: doing error downcall \n 1503 40.012862 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: Stale client: 16\n 1504 40.013326 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: \t-> closed /var/lib/nfs/rpc_pipefs/nfs/clnt16/idmap\n 1505 40.013740 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: Stale client: 17\n 1506 40.014157 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.WARNING: rpc.idmapd[4575]: \t-> closed /var/lib/nfs/rpc_pipefs/nfs/clnt17/idmap\n 1507 40.014621 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: destroying client clnt17 \n 1508 40.015082 10.100.1.35 -> 10.100.1.99 Syslog DAEMON.ERR: rpc.gssd[2498]: destroying client clnt16 \n [root@bk001 ~]#