[CentOS] Windows 2003 AD, Winbind, Kerberos and NFSv4

Sat Jul 3 01:19:28 UTC 2010
James A. Peltier <jpeltier at fas.sfu.ca>

On Fri, 2 Jul 2010, James A. Peltier wrote:

> Hi All,
>
> I'm having a bit of difficulty getting a CentOS 5.5 Kerberized NFSv4
> server working.  This server is configured as a Winbind client to a
> Windows 2003 Active Directory.  I've successfully bound it to AD and I am
> able to authenticate.  I've successfully created a NFSv4 entry in
> /etc/exports to export the /exports directory and I can successfully mount
> a non-Kerberized NFSv4 mount on a client machine.  I now want to take it
> to the next step and add Kerberos and it doesn't seem to be working for
> me.  Below are my configurations and the steps I performed.
>
> Fresh install of CentOS 5.5. Non-Kickstarted.  Wanted to get it working
> manually first.
>
>
> Used the First Boot Authentication Wizard to configure Winbind Support for
> User Information.
>
> Configured Winbind and Kerberos under the Authentication Tab
>
> Checked Local auth is sufficient and Create home directories under options
>
> The computer successfully joins the domain and appears in the Computer
> container in AD.  Below is the extract from /etc/samba/smb.conf for
> authconfig plus what I changed, everything else is stock.
>
> I disabled the firewall and SELinux is running in permissive mode on both
> the test server and test client.
>
> #======================= Global Settings =====================================
>
> [global]
> #--authconfig--start-line--
>
> # Generated by authconfig on 2010/07/01 18:32:54
> # DO NOT EDIT THIS SECTION (delimited by --start-line--/--end-line--)
> # Any modification may be deleted or altered by authconfig in future
>
>    workgroup = MY.AD.NAME
>    password server = MY.AD.SERVER
>    realm = MY.AD.NAME
>    security = ads
>    idmap uid = 16777216-33554431
>    idmap gid = 16777216-33554431
>    template shell = /bin/bash
>
> #--authconfig--end-line--
>
> #-- my additions/changes-start --
>    template homedir = /home/%U
>    winbind use default domain = true
>    winbind offline logon = true
>    winbind nested groups = true
>    winbind refresh tickets = true
>    use spnego = yes
>    use kerberos keytab = yes
> #-- my additions/changes-end --
>
> It also created an appropriate, I believe, /etc/krb5.conf to which I
> removed only the .example.com stuff resulting in the following file.
>
> [logging]
>  default = FILE:/var/log/krb5libs.log
>  kdc = FILE:/var/log/krb5kdc.log
>  admin_server = FILE:/var/log/kadmind.log
>
> [libdefaults]
>  default_realm = MY.AD.NAME
>  dns_lookup_realm = false  # with and without true tried for these
>  dns_lookup_kdc = false
>  ticket_lifetime = 24h
>  forwardable = yes
>
> [realms]
>  MY.AD.NAME = {
>   kdc = MY.AD.SERVER
>   admin_server = MY.AD.SERVER
>   kdc = MY.AD.SERVER
>  }
>
> [domain_realm]
>  my.ad.name = MY.AD.NAME
>  .my.ad.name = MY.AD.NAME
> [appdefaults]
>  pam = {
>    debug = false
>    ticket_lifetime = 36000
>    renew_lifetime = 36000
>    forwardable = true
>    krb4_convert = false
>  }
>
> If I log into this host I am properly issued a Kerberos ticket from AD so
> it would appear that Kerberos is working properly
>
> [jpeltier at oak ~]$ ssh aconite klist
> jpeltier at aconite's password:
> klist: You have no tickets cached
> Ticket cache: FILE:/tmp/krb5cc_16777216_ltvWwy
> Default principal: jpeltier at MY.AD.NAME
>
> Valid starting     Expires            Service principal
> 07/02/10 10:46:43  07/02/10 20:46:43  krbtgt/MY.AD.NAME at MY.AD.NAME
>         renew until 07/02/10 20:46:43
>
>
> Kerberos 4 ticket cache: /tmp/tkt16777216
> [jpeltier at oak ~]$
>
>
> Now I configured NFSv4 exports on the server
>
> /exports	*(rw,fsid=0)
>
> Edited /etc/sysconfig/nfs to change
>
> # Set to turn on Secure NFS mounts.
> #SECURE_NFS="yes"
>
> to
>
> # Set to turn on Secure NFS mounts.
> SECURE_NFS="yes"
>
> restarted NFS service and it appears as an export
>
> [root at aconite ~]# /etc/init.d/nfs restart
> Shutting down NFS mountd:                                  [  OK  ]
> Shutting down NFS daemon:                                  [  OK  ]
> Shutting down NFS quotas:                                  [  OK  ]
> Shutting down NFS services:                                [  OK  ]
> Shutting down RPC svcgssd:                                 [  OK  ]
> Starting RPC svcgssd:                                      [  OK  ]
> Starting NFS services:                                     [  OK  ]
> Starting NFS quotas:                                       [  OK  ]
> Starting NFS daemon:                                       [  OK  ]
> Starting NFS mountd:                                       [  OK  ]
>
> [root at aconite ~]# exportfs
> /exports        <world>
> [root at aconite ~]#
>
> To support NFSv4 with Kerberos security, we also need to generate service
> principal for NFS:
>
> [root at aconite ~]# net -U administrator ads keytab add nfs
>
> which then looks like this
>
> [root at aconite ~]# klist -k
> Keytab name: FILE:/etc/krb5.keytab
> KVNO Principal
> ---- --------------------------------------------------------------------------
>    3 host/aconite.my.ad.name at MY.AD.NAME
>    3 host/aconite.my.ad.name at MY.AD.NAME
>    3 host/aconite.my.ad.name at MY.AD.NAME
>    3 host/aconite at MY.AD.NAME
>    3 host/aconite at MY.AD.NAME
>    3 host/aconite at MY.AD.NAME
>    3 ACONITE$@MY.AD.NAME
>    3 ACONITE$@MY.AD.NAME
>    3 ACONITE$@MY.AD.NAME
>    3 nfs/aconite.my.ad.name at MY.AD.NAME
>    3 nfs/aconite.my.ad.name at MY.AD.NAME
>    3 nfs/aconite.my.ad.name at MY.AD.NAME
>    3 nfs/aconite at MY.AD.NAME
>    3 nfs/aconite at MY.AD.NAME
>    3 nfs/aconite at MY.AD.NAME
>
>
> Test on the client
>
> [root at celastrina ~]# showmount -e aconite
> Export list for aconite:
> /exports *
> [root at celastrina ~]# mount -t nfs4 aconite:/ /mnt
> [root at celastrina ~]# mount |grep -i nfs4
> aconite:/ on /mnt type nfs4 (rw,addr=199.60.1.84)
> [root at celastrina ~]#
>
> So as you can see everything is now working *without* Kerberos.  However,
> if I change the /etc/exports file on aconite to
>
> [root at aconite ~]# cat /etc/exports
> /exports        gss/krb5(rw,fsid=0)
> [root at aconite ~]# exportfs
> /exports        gss/krb5
>
>
> and then try to mount with the -o sec=krb5 on the client
>
> [root at celastrina ~]# mount -t nfs4 -o sec=krb5 aconite:/ /mnt
> mount.nfs4: Permission denied
>
> and the entry in /var/log/messages on celastrina is
>
> Jul  2 11:21:57 celastrina rpc.gssd[3302]: Using keytab file
> '/etc/krb5.keytab'
> Jul  2 11:21:57 celastrina rpc.gssd[3302]: WARNING: Failed to obtain
> machine credentials for connection to server aconite.my.ad.name
>
> nothing appears in the logs on aconite.
>
> If you've been kind enough to read this entire thread could you be kind
> enough to let me know what I am missing or doing wrong?  It seems so close
> now that I can taste it, but this one last piece doesn't seem to want to
> fall into place. :(


Hi All,

I was able to solve this one and of course, it turned out to be quite 
simple once I knew what to look for.

In order to get it to work I needed to alter the initial join of the host 
to AD.  Newer versions of samba support the option

   createupn=

In previous versions of Samba and AD you had to do this process manually 
in AD to map the server to a userPrincipleName.  Using the command

   net ads join createupn=nfs/`hostname`@<DOMAIN> -U <AD Admin User Name>

followed by

   net ads keytab add nfs -U <AD Admin User Name>

created all the appropriate info in AD and the /etc/krb5.keytab file in 
order to allow the client to mount the share via NFSv4 and krb5p 
extensions.

-bash-3.2$ mount |grep nfs4
aconite:/ on /home type nfs4 (rw,sec=krb5p,addr=199.60.1.84)

I hope this is useful to people who are trying to perform a similar 
operation. ;)


-- 
James A. Peltier
Systems Analyst (FASNet), VIVARIUM Technical Director
HPC Coordinator
Simon Fraser University - Burnaby Campus
Phone   : 778-782-6573
Fax     : 778-782-3045
E-Mail  : jpeltier at sfu.ca
Website : http://www.fas.sfu.ca | http://vivarium.cs.sfu.ca
           http://blogs.sfu.ca/people/jpeltier
MSN     : subatomic_spam at hotmail.com

TEAMWORK
  There's power in numbers.  Learn to work together.