Bug#807291: nfs-common: NFS client hangs with NFS4 mounts
Package: nfs-common
Version: 1:1.2.8-9
Severity: important
Dear Maintainer,
NFS client started to hang on NFSv4 mounts after some time of usage.
First, it mounts fine but after variable time (3 - 10 hours) of work,
all the nfs4 mounts hang and programs using them block. It's possible to
unmount them forcibly (umount -f) as I'm using soft,intr options but
it's not possible to mount them again. Mount request just hangs as well.
NFSv3 mount of some of the mounts exported via NFSv3 succeeds.
I'm using kerberos for NFSv4, but after the freeze I've tried mount with
sec=sys and sec=none and the result was the same - the mount blocking.
When I turned RPC debugging (via rpcdebug), the log ended with:
Dec 6 00:20:32 hostname kernel: [26681.878553] NFS: revalidating (0:45/1580)
Dec 6 00:20:32 hostname kernel: [26681.878562] RPC: new task initialized, procpid 10635
Dec 6 00:20:32 hostname kernel: [26681.878564] RPC: allocated task ffff880288e39f00
Dec 6 00:20:32 hostname kernel: [26681.878568] RPC: 60814 __rpc_execute flags=0x4280
Dec 6 00:20:32 hostname kernel: [26681.878573] RPC: 60814 sleep_on(queue "NFSv4.0 transport Slot table" time 4301568733)
Dec 6 00:20:32 hostname kernel: [26681.878576] RPC: 60814 added to queue ffff88029efe0298 "NFSv4.0 transport Slot table"
Dec 6 00:20:32 hostname kernel: [26681.878579] RPC: 60814 sync task going to sleep
This lead me to this bug report
https://bugzilla.redhat.com/show_bug.cgi?id=1057934 which in turn leads
to some private Red Hat bugzilla error. I wasn't able to get more
debugging info than that.
I'm not sure if it belongs to nfs-common or rather kernel, but it's NFS
specific so posting here.
-- Package-specific info:
-- rpcinfo --
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100227 2 tcp 2049
100227 3 tcp 2049
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 4 udp 2049 nfs
100227 2 udp 2049
100227 3 udp 2049
100021 1 udp 32768 nlockmgr
100021 3 udp 32768 nlockmgr
100021 4 udp 32768 nlockmgr
100021 1 tcp 32768 nlockmgr
100021 3 tcp 32768 nlockmgr
100021 4 tcp 32768 nlockmgr
100005 1 udp 32767 mountd
100005 1 tcp 32767 mountd
100005 2 udp 32767 mountd
100005 2 tcp 32767 mountd
100005 3 udp 32767 mountd
100005 3 tcp 32767 mountd
100024 1 udp 32765 status
100024 1 tcp 32765 status
-- /etc/default/nfs-common --
NEED_STATD=
STATDOPTS="--port 32765 --outgoing-port 32766"
NEED_IDMAPD=yes
NEED_GSSD=yes
RPCGSSDOPTS="-vvv"
-- /etc/idmapd.conf --
[General]
Verbosity = 3
Pipefs-Directory = /run/rpc_pipefs
Domain=domain.xy
[Mapping]
Nobody-User = nobody
Nobody-Group = nogroup
-- /etc/fstab --
/media /srv/nfs4/media none rw,rbind 0 0
/srv/home /srv/nfs4/home none rw,bind 0 0
hostname2:/multimedia /srv/multimedia nfs4 sec=krb5,soft,intr,timeo=10,retrans=10,noatime,auto,x-systemd.automount,x-systemd.device-timeout=10 0 0
-- /proc/mounts --
/dev/dm-3 /srv/nfs4/media ext4 rw,noatime,errors=remount-ro,data=ordered 0 0
/dev/mapper/vg00-home /srv/nfs4/home ext4 rw,nosuid,nodev,relatime,data=ordered 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
hostname2:/home/shared /home/shared nfs4 rw,noatime,vers=4.0,rsize=262144,wsize=262144,namlen=255,soft,proto=tcp6,port=0,timeo=10,retrans=10,sec=krb5,clientaddr=fd3c:13aa:60ba:0:ae22:bff:fe2c:b7ae,local_lock=none,addr=fd3c:13aa:60ba::1
0 0
-- System Information:
Debian Release: stretch/sid
APT prefers unstable
APT policy: (990, 'unstable'), (10, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 4.2.0-1-amd64 (SMP w/6 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=cs_CZ.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)
Versions of packages nfs-common depends on:
ii adduser 3.113+nmu3
ii initscripts 2.88dsf-59.2
ii libc6 2.21-3
ii libcap2 1:2.24-12
ii libcomerr2 1.42.13-1
ii libdevmapper1.02.1 2:1.02.110-1
ii libevent-2.0-5 2.0.21-stable-2+b1
ii libgssapi-krb5-2 1.13.2+dfsg-4
ii libk5crypto3 1.13.2+dfsg-4
ii libkeyutils1 1.5.9-8
ii libkrb5-3 1.13.2+dfsg-4
ii libmount1 2.27.1-1
ii libnfsidmap2 0.25-5
ii libtirpc1 0.2.5-1
ii libwrap0 7.6.q-25
ii lsb-base 9.20150917
ii rpcbind 0.2.3-0.2
ii ucf 3.0031
Versions of packages nfs-common recommends:
ii python 2.7.9-1
Versions of packages nfs-common suggests:
ii open-iscsi 2.0.873+git0.3b4b4500-12
pn watchdog <none>
Versions of packages nfs-kernel-server depends on:
ii libblkid1 2.27.1-1
ii libc6 2.21-3
ii libcap2 1:2.24-12
ii libsqlite3-0 3.9.2-1
ii libtirpc1 0.2.5-1
ii libwrap0 7.6.q-25
ii lsb-base 9.20150917
ii ucf 3.0031
-- Configuration Files:
/etc/default/nfs-common changed:
NEED_STATD=
STATDOPTS="--port 32765 --outgoing-port 32766"
NEED_IDMAPD=yes
NEED_GSSD=yes
RPCGSSDOPTS="-vvv"
-- no debconf information
Reply to: