There could be a better way of doing this, but what seems to work is just stopping the nfs-kernel-server service, moving all files in /var/lib/nfs/ to a different directory, touching rmtab and restarting nfs-kernel-server. exportfs -a and restart the autofs service on the client.