NFS lockd error cannot monitor client

[Alexander]

System:
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Version signature: Ubuntu 4.4.0-75.96~14.04.1-generic 4.4.59
Using NFS-common and NFS-kernel-server version 1:1.2.8-6ubuntu1.2 before kernel update and after.

After an update of a few packages (list attached), our NFS server refuses exclusive file locks after about 24h of runtime. Symptoms are that NFS clients cannot execute programs that use exclusive file locks (e.g. gradle), if using a UI like Ubuntu Desktop, the whole UI is stuck. However files can still be accessed and created via shell on a client through nfs.
Following log is produced on a client that mounts nfs:

May 10 00:46:23 irminen kernel: [2972685.514114] lockd: server pranger OK
May 10 00:47:24 irminen kernel: [2972746.611297] lockd: server pranger not responding, still trying

The nfs server produces following log:

May 8 22:30:22 pranger kernel: [6267842.076551] Btrfs loaded
May 9 04:16:02 pranger kernel: [6288582.711415] lockd: cannot monitor client
May 9 04:17:02 pranger kernel: [6288642.871345] lockd: cannot monitor client
May 9 04:18:03 pranger kernel: [6288703.031317] lockd: cannot monitor client
May 9 04:19:03 pranger kernel: [6288763.191282] lockd: cannot monitor client
May 9 04:20:03 pranger kernel: [6288823.351247] lockd: cannot monitor client
May 9 04:21:03 pranger kernel: [6288883.511292] lockd: cannot monitor client
May 9 04:22:03 pranger kernel: [6288943.671215] lockd: cannot monitor client
May 9 04:23:03 pranger kernel: [6289003.831189] lockd: cannot monitor client

Restarting the NFS Server does not resolve the issue, I had to reboot the server completely. After one day running the same problem appears again.

Mounted on 14.04 and 16.04 machines via
pranger:/srv/home /home nfs nfsvers=3 0

More info in Comment #7 (https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1689777/comments/7)

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1689777

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Alexander (ofenstichloch)]

Attached log includes the existing problem at rotation time. Then a reboot was issued.

[Alexander (ofenstichloch)]

Booting on old Kernel does not relove issue, not Kernel related so far.

[Alexander (ofenstichloch)]

Moved to nfs-kernel-server

[Alexander (ofenstichloch)]

Following output in syslog, if rpc debug level is increased by
"echo "65535" > /proc/sys/sunrpc/nlm_debug":

May 11 23:17:46 pranger kernel: [77020.767293] lockd: NSM upcall RPC failed, status=-110
May 11 23:17:46 pranger kernel: [77020.767320] lockd: cannot monitor gartenfeld
May 11 23:17:46 pranger kernel: [77020.767322] lockd: release server host gartenfeld
May 11 23:17:46 pranger kernel: [77020.767418] lockd: request from XXXXXXXXX, port=1017
May 11 23:17:46 pranger kernel: [77020.767429] lockd: LOCK called
May 11 23:17:46 pranger kernel: [77020.767432] lockd: nlmsvc_lookup_host(host='gartenfeld', vers=4, proto=tcp)
May 11 23:17:46 pranger kernel: [77020.767434] lockd: get host gartenfeld
May 11 23:17:46 pranger kernel: [77020.767435] lockd: nlmsvc_lookup_host found host gartenfeld (XXXXXXXXXX)
May 11 23:17:46 pranger kernel: [77020.767437] lockd: nsm_monitor(gartenfeld)

[Alexander (ofenstichloch)]

Added APT history of updated packages

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dean Montgomery (dmonty)]

Came across this on Ubuntu 16.04 diskless client. Fixed by enabling rpc-statd.service

sudo systemctl enable rpc-statd
sudo systemctl start rpc-statd

[Chrescht (sekateur)]

You enabled rpc-statd on the server, right?