mount.nfs: Fix minor version parsing when '-t nfs4' and '-o vers=4.x' are used together

Bug #2059197 reported by Jeff
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
nfs-utils (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
In Progress
Medium
Matthew Ruffell

Bug Description

[Impact]

When specifying both '-t nfs4' and '-o vers=4.x' on the mount command line, mount.nfs ignores any specific version set with '-o vers=4.x' or '-o vers=4,minorversion=x', and simply attempts version negotiation, which due to my recent changes in bug 2049262, would land on the highest supported nfs version, likely 4.2.

Users who specify what version they want should get the version they ask for, and thus, this is a regression introduced in 1:1.3.4-2.5ubuntu3.6.

A workaround is to drop '-t nfs4' and to use mount.nfs directly instead, i.e.:

$ sudo mount.nfs -o vers=4.1 -vvv jammy-nfs-server:/ /mnt

[Testcase]

Create two VMs, one jammy and one focal.

The jammy VM will be the server.

Server VM:
$ sudo hostnamectl set-hostname jammy-nfs-server
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install nfs-kernel-server
$ sudo mkdir /export
$ sudo mkdir /export/users
$ sudo mkdir /home/users
$ sudo vi /etc/fstab # add the following line:
/home/users /export/users none bind 0 0
$ sudo mount -a
$ sudo vi /etc/exports # add the following lines:
/export 192.168.122.0/24(rw,fsid=0,no_subtree_check,sync)
/export/users 192.168.122.0/24(rw,nohide,insecure,no_subtree_check,sync)
$ sudo systemctl restart nfs-server.service

Focal VM:
$ sudo hostnamectl set-hostname focal-nfs-client
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install nfs-common

And then try mounting with both '-t nfs4' and '-o vers=4.1':

$ sudo mount -t nfs4 -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Wed Apr 3 00:51:23 2024
mount.nfs4: trying text-based options 'vers=4.2,addr=192.168.122.217,clientaddr=192.168.122.9'

and then try with both '-t nfs4' and '-o vers=4,minorversion=1':

$ sudo mount -t nfs4 -o vers=4,minorversion=1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Wed Apr 3 02:19:03 2024
mount.nfs4: trying text-based options 'minorversion=1,vers=4.2,addr=192.168.122.217,clientaddr=192.168.122.9'

In each of those mounts, it incorrectly uses protocol version 4.2, instead of requested 4.1.

If you install the test packages in the following ppa:

https://launchpad.net/~mruffell/+archive/ubuntu/lp2059197-test

then vers should always be 4.1, instead of 4.2.

[Where problems can occur]

It is quite clear now that changing version negotiation and parsing is quite problematic, after the number of regressions the previous 1:1.3.4-2.5ubuntu3.6 caused the community, and the sheer number of commits upstream took to resolve lingering issues.

We need to be careful, to make sure we don't introduce any further regressions.

This time, we are only changing how mount.nfs parses and processes command line arguments for version specification, and not changing version negotiation itself. This could impact NFS clients, when they go to mount NFS filesystems.

If a regression were to occur, then users might have to manually mount their NFS filesystems or look into autofs. There should be no breaking changes to any behaviour when mounting NFS filesystems.

There are no changes to nfs server.

[Other Info]

This was fixed multiple times, across the following three commits:

commit 1e3ca7fe199531a372bb8e1c1033d8973097be50
Author: NeilBrown <email address hidden>
Date: Wed Jul 26 13:47:42 2017 -0400
Subject: mount: move handling of "-t nfs4" into nfs_nfs_version()
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=1e3ca7fe199531a372bb8e1c1033d8973097be50

commit 050153ef6a86ff5ab06ef65be36336824a323779
Author: NeilBrown <email address hidden>
Date: Wed Jul 26 13:51:51 2017 -0400
Subject: mount: Fix problems with parsing minorversion=
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=050153ef6a86ff5ab06ef65be36336824a323779

commit 71b807e1a69940beb31fdeb481c2e54509e8bd32
Author: Steve Dickson <email address hidden>
Date: Thu Mar 1 13:52:47 2018 -0500
Subject: mount.nfs: minorversion setting is being ignored with the -t flag
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=71b807e1a69940beb31fdeb481c2e54509e8bd32

commit 5f32083c759b468f5b0cb9302e4fe2848d37cc5b
From: Steve Dickson <email address hidden>
Date: Tue, 10 Apr 2018 10:25:40 -0400
Subject: mount.nfs: Fix auto protocol negotiation
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=5f32083c759b468f5b0cb9302e4fe2848d37cc5b

Jammy has all of these, and does not need any fixes.

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Subscribed mruffell who authored that particular SRU, for consideration.

tags: added: regression-update
Revision history for this message
Matthew Ruffell (mruffell) wrote :

Hi Jeff,

I believe this is a valid bug. I can reproduce it:

On Focal:

$ sudo mount -t nfs4 -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Wed Apr 3 00:51:23 2024
mount.nfs4: trying text-based options 'vers=4.2,addr=192.168.122.217,clientaddr=192.168.122.9'

On Jammy:

$ sudo mount -t nfs4 -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Wed Apr 3 00:57:08 2024
mount.nfs4: trying text-based options 'vers=4.1,addr=192.168.122.217,clientaddr=192.168.122.213'

I apologise for introducing this particular regression when I changed the
version negotiation. Let me try and find a fix and build a test package.

In the meantime, you can use a workaround of dropping "-t nfs4" and changing
"mount" to "mount.nfs", i.e.:

$ sudo mount.nfs -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Wed Apr 3 00:51:59 2024
mount.nfs: trying text-based options 'vers=4.1,addr=192.168.122.217,clientaddr=192.168.122.9'

I will write back shortly.

Thanks,
Matthew

Changed in nfs-utils (Ubuntu):
status: New → Fix Released
Changed in nfs-utils (Ubuntu Focal):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Matthew Ruffell (mruffell)
Revision history for this message
Matthew Ruffell (mruffell) wrote :

Attached is a debdiff for Focal which fixes this issue.

summary: - Issue with nfs4.0 mounts after upgrade to 3.6
+ Fix minor version parsing when '-t nfs4' and '-o vers=4.x' are used
+ together
summary: - Fix minor version parsing when '-t nfs4' and '-o vers=4.x' are used
- together
+ mount.nfs: Fix minor version parsing when '-t nfs4' and '-o vers=4.x'
+ are used together
description: updated
tags: added: seg
Revision history for this message
Matthew Ruffell (mruffell) wrote :

Hi Jeff,

Again, I apologise for introducing this regression, but we are on our way to getting this fixed.

The fixes are:

commit 1e3ca7fe199531a372bb8e1c1033d8973097be50
Author: NeilBrown <email address hidden>
Date: Wed Jul 26 13:47:42 2017 -0400
Subject: mount: move handling of "-t nfs4" into nfs_nfs_version()
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=1e3ca7fe199531a372bb8e1c1033d8973097be50

commit 050153ef6a86ff5ab06ef65be36336824a323779
Author: NeilBrown <email address hidden>
Date: Wed Jul 26 13:51:51 2017 -0400
Subject: mount: Fix problems with parsing minorversion=
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=050153ef6a86ff5ab06ef65be36336824a323779

commit 71b807e1a69940beb31fdeb481c2e54509e8bd32
Author: Steve Dickson <email address hidden>
Date: Thu Mar 1 13:52:47 2018 -0500
Subject: mount.nfs: minorversion setting is being ignored with the -t flag
Link: https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commitdiff;h=71b807e1a69940beb31fdeb481c2e54509e8bd32

I have built these into a test package, if you are interested in taking a look.

Please note this package is NOT SUPPORTED by Canonical, and is for TESTING
PURPOSES ONLY. ONLY Install in a dedicated test environment.

Instructions to install (On a Focal system):
1) sudo add-apt-repository ppa:mruffell/lp2059197-test
2) sudo apt update
3) sudo apt install nfs-common
4) sudo apt-cache policy nfs-common | grep Installed
Installed: 1:1.3.4-2.5ubuntu3.6+lp2059197v20240403b1

If you try mount your share with the version specified, it should work.

I'll write back once we have a package in -proposed with the fix.

Thanks,
Matthew

Revision history for this message
Jeff (jeffmbarnes) wrote :

Hi Matthew,

Its no problem, I can see how changing the version negotiation would be tricky, thanks for looking at it so quickly.

Using the workaround and everything working correctly, no dramas

Thanks again,
Jeffrey

Revision history for this message
Heitor Alves de Siqueira (halves) wrote :

Hi Matthew,

thanks for the quick follow-up on the regression! Being the sponsor for the original patch in bug 2049262, I wanted to give this one some deeper attention. Version parsing seems to have been a difficult area upstream, with several follow-up fixes indeed, so thank you for detailing the required commits.

I've done a double-check upstream, and noticed an additional fix that doesn't seem to be on your list:

https://git.linux-nfs.org/?p=steved/nfs-utils.git;a=commit;h=5f32083c759b
- 5f32083c mount.nfs: Fix auto protocol negotiation

$ git describe --contains 5f32083c759b
nfs-utils-2-3-2-rc2~9

This fixes a regression with falling back to NFSv3 on servers that don't support v4, and was introduced by 71b807e1. Could you please confirm whether we need to pull it in with the ones you already listed for Focal? It's already present in Jammy and newer.

Changed in nfs-utils (Ubuntu Focal):
status: In Progress → Incomplete
Revision history for this message
Matthew Ruffell (mruffell) wrote :

Attached is a new debdiff which includes the missing patch. I did another look upstream, but I think we are okay now.

Changed in nfs-utils (Ubuntu Focal):
status: Incomplete → In Progress
description: updated
Revision history for this message
Matthew Ruffell (mruffell) wrote :
Download full text (8.6 KiB)

Sanity testing the test package:

https://launchpad.net/~mruffell/+archive/ubuntu/lp2059197-test

nfs-common 1:1.3.4-2.5ubuntu3.6+lp2059197v20240411b1

Testcase for this bug, i.e. the regression.

$ sudo mount -t nfs4 -o vers=4.2 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:14:52 2024
mount.nfs4: trying text-based options 'vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'

$ sudo mount -t nfs4 -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:10:26 2024
mount.nfs4: trying text-based options 'vers=4.1,addr=192.168.122.14,clientaddr=192.168.122.249'

$ sudo mount -t nfs4 -o vers=4.0 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:15:02 2024
mount.nfs4: trying text-based options 'vers=4.0,addr=192.168.122.14,clientaddr=192.168.122.249'

$ sudo mount -t nfs4 -o vers=4,minorversion=2 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:15:48 2024
mount.nfs4: trying text-based options 'vers=4,vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'

$ sudo mount -t nfs4 -o vers=4,minorversion=1 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:13:33 2024
mount.nfs4: trying text-based options 'vers=4,vers=4.1,addr=192.168.122.14,clientaddr=192.168.122.249'

$ sudo mount -t nfs4 -o vers=4,minorversion=0 -vvv jammy-nfs-server:/ /mnt
mount.nfs4: timeout set for Thu Apr 11 05:15:54 2024
mount.nfs4: trying text-based options 'vers=4,vers=4.0,addr=192.168.122.14,clientaddr=192.168.122.249'

Looks OK.

Testcase for the original bug, 2049262:

$ sudo cat /proc/fs/nfsd/versions
-2 +3 +4 +4.1 +4.2

$ sudo mount -o vers=4 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:17:55 2024
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'
$ sudo umount /mnt
$ sudo mount -o vers=4.2 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:18:01 2024
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'
$ sudo umount /mnt
$ sudo mount -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:18:05 2024
mount.nfs: trying text-based options 'vers=4.1,addr=192.168.122.14,clientaddr=192.168.122.249'
$ sudo umount /mnt
$ sudo mount -o vers=4.0 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:18:12 2024
mount.nfs: trying text-based options 'vers=4.0,addr=192.168.122.14,clientaddr=192.168.122.249'

Looks OK.

$ sudo cat /proc/fs/nfsd/versions
-2 +3 +4 -4.0 +4.1 +4.2

$ sudo mount -o vers=4 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:20:40 2024
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'
$ sudo umount /mnt
$ sudo mount -o vers=4.2 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:20:47 2024
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.122.14,clientaddr=192.168.122.249'
$ sudo umount /mnt
$ sudo mount -o vers=4.1 -vvv jammy-nfs-server:/ /mnt
mount.nfs: timeout set for Thu Apr 11 05:20:53 2024
mount.nfs: trying text-based options 'vers=4.1,addr=192.168.122.14,clientaddr=192.168.122.2...

Read more...

Revision history for this message
Matthew Ruffell (mruffell) wrote :

Thanks for pointing out the missing patch Heitor. I was indeed missing it.

Please review and sponsor the V2 debdiff.

Thanks,
Matthew

Revision history for this message
Heitor Alves de Siqueira (halves) wrote :

Thanks for the revised debdiff, Matthew! And nice work on the extensive sanity check for the version laddering.

The new debdiff looks good, I've sponsored it for Focal. Thanks!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.