Kerberized SSH with homedirs on Kerberized NFS
Prev Part I.  Miscellaneous subjects (and those still to be sorted)  Next

Kerberized SSH with homedirs on Kerberized NFS

Jurjen Bokma

Last modified: "2018-06-20 16:13:54 apprentice"

Abstract

We have an SSH server with users' home directories on kerberized NFS. So authenticating to SSH using Kerberos is nice. In addition, the client side Kerberos credentials are passed to the server, where they should be used to obtain access to $HOME.

    • Kerberos is configured to work on the ssh server as well as the client. You have a principal, and can get credentials for it:

      apprentice@client:~$ kinit apprentice
      apprentice@MYDOMAIN.COM's Password:
      apprentice@client:~$ klist
      Credentials cache: FILE:/tmp/krb5cc_1000
      Principal: apprentice@MYDOMAIN.COM

      Issued                Expires               Principal
      Apr 14 14:59:19 2016  Apr 15 00:59:16 2016  krbtgt/MYDOMAIN.COM@MYDOMAIN.COM

    • Both server and client are members of a Kerberos realm, i.e. they have host keys:

      apprentice@client:~$ sudo ktutil list
      FILE:/etc/krb5.keytab:

      Vno  Type                     Principal                          Aliases
        3  aes256-cts-hmac-sha1-96  192.168.139.218$@MYDOMAIN.COM

      The realm in our case is a MS Active Directory domain.

      [Note]Note

      The principal is not the Kerberos-conventional fqdn. It's an IP number with a dollar sign appended. This happens more often in MS domains, and it has important consequences below.

    • The server holds Kerberos keys for host/<hostname>@<REALM>:

      apprentice@ssh-server:~$ sudo ktutil list|grep host
            111  aes256-cts-hmac-sha1-96  host/ssh-server.mydomain.com@MYDOMAIN.COM

  2. /etc/ssh/sshd_config should look like this: 

    # Main settings
#
Port 22

HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key

# Settings disabled for security
#
StrictModes                      yes
PermitRootLogin                  no
PermitEmptyPasswords             no
HostbasedAuthentication          no

# Kerberos-related stuff
GSSAPIAuthentication             yes
GSSAPIKeyExchange                yes 1
GSSAPIStoreCredentialsOnRekey    yes 2
GSSAPICleanupCredentials         yes 3

# Debian/ubuntu defaults which are different from upstream sshd defaults
#
Protocol                         2
ChallengeResponseAuthentication  no
PrintMotd                        no
AcceptEnv                        LANG LC_*
Subsystem sftp                   /usr/lib/openssh/sftp-server
Banner                           /etc/issue.net
UsePAM                           yes
X11Forwarding                    yes
X11UseLocalhost                                  no

# Authentication mechanisms
#
PasswordAuthentication           no
PubkeyAuthentication             no

# Other custom settings
#
AllowAgentForwarding             yes
MaxStartups                      10:30:60
AuthorizedKeysFile               /var/lwp/ssh/%u/authorized_keys %h/.ssh/authorized_keys 4

# Restricted logins (if any)
#

# These hosts are allowed to use Kerberos
Match Address 192.168.0.0/16 5
GSSAPIAuthentication yes
PubkeyAuthentication yes
KerberosAuthentication yes
AuthenticationMethods publickey,gssapi-keyex publickey,gssapi-with-mic publickey,password 6

    1

    Without GSSAPIKeyExchange, GSSAPIStoreCredentialsOnRekey doesn't seem to work.

    2

    Kerberos by default uses short-lived tickets, but they may be renewed for a much longer time. When such a renewal happens on the client, the new key must also become available to the server. Doing this upon rekeying of the connection is a trick, but one that works.

    3

    When the user logs out from the server, his credentials must not linger there.

    [Warning]Warning

    When the user has multiple ssh session to the server, this may delete credentials that are still in use by another session.

    4

    This has little to do with Kerberos, but it's useful. When the user logs in on the server, the ssh daemon, in order to verify the user's ssh keys, will by default want to access ~/.ssh/authorized_keys. But it cannot, because $HOME is on a kerberized NFS4 share, and in our configuration, root doesn't have read access there. So we store the authorized_keys elsewhere.

    5

    Our Kerberos server is not open to the world, so we only allow users to authenticate with Kerberos from where they can reach the Kerberos server and obtain a ticket.

    6

    This setting is tricky, and I don't understand it fully. The meaning according to the docs is to allow either of the three comma-separated, ordered sets of two authentication methods.

    But change the order of publickey and gssapi-keyex, and it doesn't work any more. I suppose that some lookups done for the public key aso work for the keyex, but not the other way around.

    Also, gssapi-keyex authenticates hosts, but not users. It is apparently built-in mandatory that users do get authenticated, because gssapi-mic must also succeed, even if gssapi-keyex worked. And without the gssapi-keyex in this list, key exchange is forbidden, which leads to credentials not being updated, which in turn leads to access to $HOME being soon denied.

    [Note]Note

    It is also possible to specify: 

    AuthenticationMethods publickey,gssapi-keyex publickey,gssapi-with-mic publickey password

    In that case, after succesful public key authentication, both gssapi-keyex and gssapi-with-mic will be attempted, giving the full benefit of Kerberos authentication and credential-passing. Yet if public key authentication fails, password authentication on its own may still succeed.

    Of course you need to reload the ssh server:

    apprentice@server:~$ sudo service ssh reload

    Maybe you even want to run it in the foreground for test:

    apprentice@server:~$ sudo service ssh stop
    apprentice@server:~$ /usr/sbin/sshd -D -d

  3. You want this in ~/.ssh/config: 

    ForwardX11 yes

host ssh-server.mydomain.com ssh-server 192.168.32.119 1
      HostName ssh-server.mydomain.com 2
      GSSAPIAuthentication yes
      GSSAPIKeyExchange yes
#     GSSAPIClientIdentity 192.168.139.218$ 3
#     GSSAPIServerIdentity ssh-server.mydomain.com 4
      GSSAPIDelegateCredentials yes 5
      GSSAPIRenewalForcesRekey yes 6
#     GSSAPITrustDns yes 7

    1

    This host specification matches any name by which the host may be called...

    2

    ... but the HostName is very specific: the FQDN. This is used by Kerberos to know for which host to obtain a ticket. And if the ticket is wrong about that, kerberos authentication will fail.

    3

    Kerberos assumes the client has a principal that uses its FQDN. In some MS Active Directory based realms, like ours, it doesn't, and the IP number is used with a '$' attached. Yet it works without this setting.

    4

    Same as ClientIdentity, but now for the server.

    5

    This setting is crucial. It allows the server to use the client's credentials, e.g. to obtain a ticket for NFS server side.

    6

    When the credentials get renewed client side, also notify the server. If you don't set this, credentials will rapidly expire on the server.

    7

    Instead of specifying the hostname manually, you could trust DNS.

  4. apprentice@client:~$ ssh apprentice@ssh-server
    <snip>
    Last login: Thu Apr 14 15:31:45 2016 from client.mydomain.com
    apprentice@ssh-server:~$ klist
    Credentials cache: FILE:/tmp/krb5cc_1001_6MxnQ5kmIX
    Principal: apprentice@MYDOMAIN.COM

    Issued                Expires               Principal
    Apr 14 15:31:55 2016  Apr 15 00:59:16 2016  krbtgt/MYDOMAIN.COM@MYDOMAIN.COM

    [Note]Note

    There is no NFS ticket, as I had logged in just before. When I logged out, the previous credentials were destroyed, but the kernel lags behind in discovering that they have gone. In a while, it will find out there are no credentials, but it'll then use my credentials to get a new ticket, and access $HOME for me anyway.

  5. apprentice@client:~$ klist
    Credentials cache: FILE:/tmp/krb5cc_1000
    Principal: apprentice@MYDOMAIN.COM

    Issued                Expires               Principal
    Apr 14 14:59:19 2016  Apr 15 00:59:16 2016  krbtgt/MYDOMAIN.COM@MYDOMAIN.COM
    Apr 14 15:31:44 2016  Apr 15 00:59:16 2016  host/ssh-server.mydomain.com@MYDOMAIN.COM
    apprentice@client:~$ krenew -v
    krenew: renewing credentials for apprentice@MYDOMAIN.COM
    apprentice@client:~$ klist
    Credentials cache: FILE:/tmp/krb5cc_1000
    Principal: apprentice@MYDOMAIN.COM

    Issued                Expires               Principal
    Apr 14 15:37:22 2016  Apr 15 01:37:19 2016  krbtgt/MYDOMAIN.COM@MYDOMAIN.COM

    See that the server-side ticket has been updated:

    apprentice@ssh-server:~$ klist
    Credentials cache: FILE:/tmp/krb5cc_1001_6MxnQ5kmIX
    Principal: apprentice@MYDOMAIN.COM

    Issued                Expires               Principal
    Apr 14 15:39:22 2016  Apr 15 01:37:19 2016  krbtgt/MYDOMAIN.COM@MYDOMAIN.COM

  6. Run the SSH server in the foreground, with debugging on:

    sudo /usr/sbin/sshd -D -dd

    Run the client verbosely:

    ssh -vvv apprentice@ssh-server

    Chances are that your server doesn't have the correct host keys. Perhaps you need to add one:

    sudo msktutil --update --enctypes 16 --computer-name 192.168.32.119$ --realm MYDOMAIN.COM --base OU=PCS,OU=workplaces,OU=MYDOMAIN,DC=mydomain,DC=com -s host/ssh-server.mydomain.com