Bug#775541: tests of new packages
Hello Vincent McIntyre.
Thanks for your thurough testing and useful feedback.
Let me start with a disclaimer: I'm not maintaining nfs (and I'm
not even using it myself so my knowledge is very limited). My only
involvement here is trying to squash some RC bugs and unblocking
work elsewhere by importing native systemd units.
On Fri, Sep 09, 2016 at 11:57:34AM +1000, Vincent McIntyre wrote:
>
> Thanks Andreas for those new packages.
> I did some testing of the 1.2.8-9.2 packages on a clean jessie install.
> They are pretty close but I found an issue with NFS exports in one case.
>
> I used the attached check.sh script to show the state of various targets
> as I changed things. The attached results.tar shows the output from it.
> Between each major stage below, the system is rebooted.
> Then I log in and run check.sh before doing anything else.
> Throughout, /etc/network/interfaces contained:
> source /etc/network/interfaces.d/*
> auto lo
> iface lo inet loopback
> allow-hotplug eth0
I think this ^^^^ is problematic and unfortunately spoils my interest
in the rest of your testing.
>From what I've been told one of the main differences between "auto"
and "allow-hotplug" is that allow-hotplug will not block your bootup
so services will run wild and start up ASAP. This means they can and
likely will start up before your network connection is up. Specially
if using dhcp which takes time to configure the interface and extra
so if you're using a network device (eg. usb dongle) which takes a
long time to initialize.
I assume eth0 was involved in your tests, and if so it would be
very useful to know what difference it makes if you use auto
instead.
> iface eth0 inet dhcp
> and /etc/network/interfaces.d/ was empty.
>
> step 1 - jessie system with nfs packages 1.2.8-9
> one fstab mount, no exports. no changes to /etc/default/nfs-*.
> The fstab mount works ok; it tries to mount it way too early
> but we invoke it with the 'bg' option so eventually it works.
>
> step 2 - as above but with the _netdev option added to the fstab entry.
> System still tries to mount too early, before dhclient is done,
> but it works in the end.
>
> step 3 - as above but export one filesystem.
> The fstab mount works, as above.
> Tried a few different ways to do the exports
> - wildcard host (step3.1)
> This works ok.
> - point the export at a single host (step3.2)
> This tries to start mountd to early, before it can resolve
> the hostname given in the exports file.
> As a result we end up with an empty export list:
> # showmount -e
> Export list for install:
> #
> Restarting the service once the system is fully booted works
> # systemctl restart nfs-kernel-server.service
> # showmount -e
> Export list for install:
> /data/INSTALL_1 mayhem.atnf.CSIRO.AU
> #
> - point the export at a netgroup (step3.3)
> This works, even though the netgroup cannot be resolved
> (there's no definition for it).
> # showmount -e
> Export list for install:
> /data/INSTALL_1 @all_hosts
> #
>
> NB nfs-kernel-server starts before dhclient
> Sep 08 17:04:46 install nfs-kernel-server[657]: Exporting directories for NFS kernel daemon....
> Sep 08 17:04:46 install kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
> Sep 08 17:04:46 install kernel: NFSD: starting 90-second grace period (net ffffffff818ba280)
> ...
> Sep 08 17:04:46 install rpc.mountd[738]: Version 1.2.8 starting
> Sep 08 17:04:46 install nfs-kernel-server[657]: Starting NFS kernel daemon: nfsd mountd.
> Sep 08 17:04:46 install acpid[734]: starting up with netlink and the input layer
> Sep 08 17:04:46 install acpid[734]: 1 rule loaded
> Sep 08 17:04:46 install acpid[734]: waiting for events: event logging is off
> ...
> Sep 08 17:04:49 install kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
> Sep 08 17:04:49 install kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> Sep 08 17:04:50 install dhclient[532]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 6
> ...
> Sep 08 17:04:56 install ifup[522]: Starting rpcbind (via systemctl): rpcbind.service.
> Sep 08 17:04:56 install ifup[522]: Starting nfs-common (via systemctl): nfs-common.service.
> Sep 08 17:04:56 install kernel: Key type dns_resolver registered
> Sep 08 17:04:56 install kernel: NFS: Registering the id_resolver key type
> Sep 08 17:04:56 install kernel: Key type id_resolver registered
> Sep 08 17:04:56 install kernel: Key type id_legacy registered
>
>
> step3.4 - install new nfs packages but don't reboot yet.
> I didn't analyse this step carefully
>
> step 4 - jessie system with nfs packages 1.2.8-9.2
> one fstab mount, one exported directory.
>
> rpc.mountd fails to start because it wants rpcbind to be there,
> but rpcbind has not been started yet. showmount -e fails:
> # showmount -e
> clnt_create: RPC: Program not registered
>
> step4.1 modify /lib/systemd/system/nfs-mountd.service,
> - After=network.target local-fs.target
> + After=rpcbind.target network.target local-fs.target
> run systemd daemon-reload
> but don't reboot.
> The exports are still not there
> # showmount -e
> clnt_create: RPC: Program not registered
>
> re run exports, it fails
> # exportfs -rav
> exporting @all_hosts:/data/INSTALL_1
> # showmount -e
> clnt_create: RPC: Program not registered
> #
>
> step 5 - reboot after step4.1:
>
> usr-local.mount starts a little (1-2s) later. It starts
> just after ifup, but before dhclient has gotten an answer
> so it can't resolve the server hostname, but eventually
> the mount does work.
>
> Exports also work, in some cases:
> - wildcard case is OK (step5.1)
> # showmount -e
> Export list for install:
> /data/INSTALL_1 *
>
> - single-host case FAILS (step5.2)
> # showmount -e
> Export list for install:
> #
>
> - export-to-a-netgroup case is OK (step5.3).
> # showmount -e
> Export list for install:
> /data/INSTALL_1 @all_hosts
>
> In the failing case,
> rpc.mountd starts after ifup and rpcbind,
> but before dhclient is done.
> It can't resolve the hostname of the client system,
> so the export is not performed and I end up with
> # showmount -e
> Export list for install:
> #
> If I log in and run 'systemctl restart nfs-server.service'
> when the system is fully booted, the export succeeds
> # showmount -e
> Export list for install:
> /data/INSTALL_1 client.f.q.d.n
>
> step 6 - as step 5, but with a random 30s wait added to
> the end of /etc/default/nfs-kernel-server.
>
> That allowed the export to a single host to work.
> # showmount -e
> Export list for install:
> /data/INSTALL_1 client.f.q.d.n
>
> It works because all the nfs bits have to wait on
> nfs-config.service which now takes 30s to complete.
> This gives dhclient plenty of time to finish its setup.
Hopefully with "auto" you should not need this workaround.
>
> Sep 08 17:50:08 install kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
> Sep 08 17:50:08 install dhclient[531]: Internet Systems Consortium DHCP Client 4.3.1
> Sep 08 17:50:08 install dhclient[531]: Copyright 2004-2014 Internet Systems Consortium.
> Sep 08 17:50:08 install dhclient[531]: All rights reserved.
> Sep 08 17:50:08 install dhclient[531]: For info, please visit https://www.isc.org/software/dhcp/
> Sep 08 17:50:08 install dhclient[531]:
> Sep 08 17:50:08 install ifup[525]: Internet Systems Consortium DHCP Client 4.3.1
> Sep 08 17:50:08 install ifup[525]: Copyright 2004-2014 Internet Systems Consortium.
> Sep 08 17:50:08 install ifup[525]: All rights reserved.
> Sep 08 17:50:08 install ifup[525]: For info, please visit https://www.isc.org/software/dhcp/
> Sep 08 17:50:08 install rpcbind[530]: Starting rpcbind daemon....
> Sep 08 17:50:08 install dhclient[531]: Listening on LPF/eth0/de:ad:be:ef:5b:e7
> ...
> Sep 08 17:50:11 install kernel: e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
> ...
> Sep 08 17:50:13 install ifup[525]: bound to 1.2.3.4 -- renewal in 824880 seconds.
> Sep 08 17:50:13 install ifup[525]: Starting rpcbind (via systemctl): rpcbind.service.
> Sep 08 17:50:13 install kernel: FS-Cache: Loaded
> Sep 08 17:50:13 install kernel: FS-Cache: Netfs 'nfs' registered for caching
> Sep 08 17:50:13 install kernel: Key type dns_resolver registered
> Sep 08 17:50:13 install kernel: NFS: Registering the id_resolver key type
> Sep 08 17:50:13 install kernel: Key type id_resolver registered
> Sep 08 17:50:13 install kernel: Key type id_legacy registered
> Sep 08 17:50:38 install rpc.statd[603]: Version 1.2.8 starting
> Sep 08 17:50:38 install rpc.statd[603]: Flags: TI-RPC
> Sep 08 17:50:38 install rpc.mountd[608]: Version 1.2.8 starting
> Sep 08 17:50:38 install systemd[1]: usr-local.mount: Directory /usr/local to mount over is not empty, moun
> Sep 08 17:50:38 install kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
> Sep 08 17:50:38 install kernel: NFSD: starting 90-second grace period (net ffffffff818ba280)
> Sep 08 17:50:38 install sm-notify[628]: Version 1.2.8 starting
>
> I tried messing around with depending on the nss-lookup.target
> to see if I could make rpc.mountd wait a bit longer (until it
> could resolve hostnames) but I was unable to do make it work.
>
> I hope all this is of some help.
>
> --
Regards,
Andreas Henriksson
Reply to: