AIX 7.3 NFS client frequently returns an EIO error to an application when reading or writing to a file that has been locked with fcntl() on a Ubuntu 20.04 NFSV4 server
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
New
|
Undecided
|
Ubuntu on IBM Power Systems Bug Triage |
Bug Description
---Problem Description---
AIX 7.3 NFS client frequently returns an EIO error to an application when reading or writing to a file that has been locked with fcntl(). NFS server is Ubuntu 20.04.6 LTS, GNU/Linux 5.4.0-139-generic x86_64. The problem does not appear to affect other combinations of NFS client (including AIX 7.2) with this NFS server.
The AIX team have indicated that the cause of the EIO is triggered by the NFS server returning a BAD_SEQID error which leads to the AIX NFS client incorrectly zeroing the stateid, which then leads to the NFS server returning a BAD_STATEID error and the NFS client then returns the EIO error. The AIX team would like to understand why the BAD_SEQID has been returned.
---uname output---
Linux duckseason 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Machine Type = VMware ESXi Server 7.0 4 x Intel(R) Xeon(R) Gold 6348H CPU @ 2.30GHz
---Steps to Reproduce---
We cannot offer a simple way to recreate the problem as it involves IBM MQ running on two primary machines (AIX) using the Ubuntu server for it's HA NFSv4 storage.
However, we can provide any requested trace or dumps from any or all of the involved machines.
tags: | added: architecture-x8664 bugnameltc-203937 severity-high targetmilestone-inin2004 |
Changed in ubuntu: | |
assignee: | nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) |
affects: | ubuntu → linux (Ubuntu) |
Hi, reading the bug description one of the first things that caught my attention is that the kernel (5.4.0-156) seems to be a bit outdated, since the latest (aot) is 5.4.0.166.
Would you mind retrying this with the latest kernel (ideally actually with the latest userspace, for example after having done a 'apt update' and 'apt full-upgrade'), since there will be a difference of hundreds of patches (also upstream stable) between these.
On top I believe it would probably very helpful to have rpcdebug enabled for the NFS Server, like:
rpcdebug -m nfsd -s all
rpcdebug -m nlm -s all
rpcdebug -m rpc -s all
Btw. it would also be interesting to know it this also happens with a bare-metal install of the NFS server, means without having VMware in between (avoiding any potential flaws with VMware virtual network components, like virtual switches.).
I think technically this is not a Launchpad bug for the Ubuntu on IBM Power project, since here Ubuntu runs on amd64 (and on VMware), but we may still try to figure out what's going on.