NFS4 delegations (Squeeze server, Lucid client): negative speedup

Jurjen Bokma

March 2012


We have a Debian Squeeze server, and an Ubuntu Lucid client. This is the line in /etc/fstab on the client that matters:

192.168.63.131:/lwphome  /home nfs4 noauto,sec=krb5i,rsize=32768,wsize=32768,clientaddr=192.168.63.131
    

Kerberos and LDAP combine to do authentication and lookup. Ca. 50,000 users are in the database. Everything works, but read and write delegations don't, as you can see on the server:

apprentice@server:~# cat /proc/slabinfo|egrep '^#|nfsd4_delegations'
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
nfsd4_delegations      0      0    264   31    2 : tunables    0    0    0 : slabdata      0      0      0

... and on the client:

apprentice@client:~$ cat /proc/self/mountstats|grep DELEGRETURN
    DELEGRETURN: 0 0 0 0 0 0 0 0

So let's make them work.

  1. After much searching, in which this mail by David V. Cloud I found was helpful, it turned out that the Lucid kernel is just too old. We upgrade to a backported Oneiric kernel:

    apprentice@client:~# sudo apt-get install -y linux-image-generic-lts-backport-oneiric linux-headers-generic-lts-backport-oneiric

  2. The executable rpc.svcgssd, which must run, is in the package nfs-kernel-server. (That's already been mentioned as a bug.) So we install:

    apprentice@client:~$ sudo apt-get install nfs-kernel-server

  3. Of course, we don't want to actually run the NFS server on the client. These measures don't turn it off completely, just make it pipe down a bit:

    apprentice@client:~$ sudo sed -i '/^NEED_SVCGSSD=$/ s/^NEED_SVCGSSD=$/NEED_SVCGSSD=yes/g' /etc/default/nfs-kernel-server
    apprentice@client:~$ sudo sed -i '/^RPCMOUNTDOPTS=--manage-gids$/ s/^RPCMOUNTDOPTS=--manage-gids*$/RPCMOUNTDOPTS="--manage-gids -t 0"/g' /etc/default/nfs-kernel-server
    apprentice@client:~$ sudo sed -i '/^RPCNFSDCOUNT=.*$/ s/^RPCNFSDCOUNT=.*$/RPCNFSDCOUNT=0/g' /etc/default/nfs-kernel-server

  4. I read somewhere that the client needed the server in /etc/hosts, and I forgot to remove that again, so I mention it here.

  5. I didn't bother to find out whether restarting the NFS services in a certain order would work or not. Just rebooted the client.

  6. Now delegations work:

    apprentice@server:~# cat /proc/slabinfo|egrep '^#|nfsd4_delegations'
    # name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
    nfsd4_delegations    310    310    264   31    2 : tunables    0    0    0 : slabdata     10     10      0

    ... and on the client:

    apprentice@client:~$ cat /proc/self/mountstats|grep DELEGRETURN
       DELEGRETURN: 2184 2184 0 524160 681408 46 1014 1101

However, delegations don't speed all things up. For testing, I untarred GNU tar, and did the usual ./configure, make. Between the Lucid and Oneiric kernel, there was hardly any difference. With delegations enabled however, untar went from 8 to 45 seconds, configure went from 90 to 200 seconds, and make went from 27 to 31 seconds. But then again, maybe software building isn't a good test.

So far for delegations.

[Note]Note

The server doesn't handle client-side replacement of client Kerberos keys well. When I reinstalled a client, delegations failed because the server kept sending messages which used the principal of the old, overwritten client. A server reboot fixed that.