Merge nfs-utils from Debian experimental for 22.04 - version in Ubuntu is *very* old

[Dan Streetman]

nfs-utils has fallen FAR behind upstream, in both Debian and Ubuntu; we've had 1.3.4 since Bionic:

$ rmadison -u debian nfs-utils
nfs-utils | 1:1.2.8-9 | oldoldstable | source
nfs-utils | 1:1.3.4-2.1 | oldstable | source
nfs-utils | 1:1.3.4-2.5 | stable | source
nfs-utils | 1:1.3.4-3 | testing | source
nfs-utils | 1:1.3.4-3 | unstable | source
nfs-utils | 1:1.3.4-3 | unstable-debug | source
$ rmadison nfs-utils
 nfs-utils | 1:1.2.5-3ubuntu3 | precise | source
 nfs-utils | 1:1.2.5-3ubuntu3.2 | precise-updates | source
 nfs-utils | 1:1.2.8-6ubuntu1 | trusty | source
 nfs-utils | 1:1.2.8-6ubuntu1.2 | trusty-updates | source
 nfs-utils | 1:1.2.8-9ubuntu12 | xenial | source
 nfs-utils | 1:1.2.8-9ubuntu12.2 | xenial-updates | source
 nfs-utils | 1:1.3.4-2.1ubuntu5 | bionic | source
 nfs-utils | 1:1.3.4-2.1ubuntu5.2 | bionic-updates | source
 nfs-utils | 1:1.3.4-2.3ubuntu3 | disco | source
 nfs-utils | 1:1.3.4-2.5ubuntu2 | eoan | source
 nfs-utils | 1:1.3.4-2.5ubuntu3 | focal | source
 nfs-utils | 1:1.3.4-2.5ubuntu3 | groovy | source

Version 1.3.4 is from 2016, so we are 4 years behind upstream.

https://kernel.org/pub/linux/utils/nfs-utils/

Debian has an 'upstream' branch:
https://salsa.debian.org/kernel-team/nfs-utils/-/tree/upstream

as well as a WIP MR:
https://salsa.debian.org/kernel-team/nfs-utils/-/merge_requests/1

and a newer WIP MR:
https://salsa.debian.org/kernel-team/nfs-utils/-/merge_requests/3

Someone should get nfs-utils updated in Debian and Ubuntu.

[Dan Streetman (ddstreet)]

@seb218 I don't know if you had specific reasoning for setting importance to 'low' but i think this is actually getting pretty critical, to be approaching 6 years behind upstream.

[Robie Basak (racb)]

Is there a specific reason this needs updating, apart from the version number being higher? For example are there any missing features, bugs fixed, or incompatibilities reduced?

[Dan Streetman (ddstreet)]

> Is there a specific reason this needs updating, apart from the version number being higher? For example are there any missing features, bugs fixed, or incompatibilities reduced?

That's a great question, and I would defer to the Foundations team to evaluate that since, IIUC, they are the package maintainer and SME.

A trivial check of the number of commits and line changes shows:

$ git log --oneline nfs-utils-1-3-4..master | wc -l
515
$ git diff nfs-utils-1-3-4..master | wc -l
48365

[weisun (wsun07)]

Hi Dan,

Yes, there is a reason. The mount nfs with sloppy (-s) option is broken in Debian 11 release. I just checked the mainline and confirmed it has been fixed with the following two changes.

2021-07-26 Steve Dickson mount.nfs: Fix the sloppy option processing
2021-06-10 Jianhong Yin mount.nfs: insert 'sloppy' at beginning of the options

If you do not want to update the version, would you please at least merge these two changes into Debian package? I am happy to share the changes that merged by me. I confirmed it works. It took me one week to figure it out and hope others will not repeat what I have done.

Thanks,
Wei

[Sergio Durigan Junior (sergiodj)]

As can be seen in the Debian bug, the nfs-utils package has been updated there and will be merged into Ubuntu during the next cycle (JJ).

[Bryce Harrington (bryce)]

Possibly this could be just a sync, however the old version has a fair bit of delta that will need to be reviewed:

nfs-utils (1:1.3.4-6ubuntu1) impish; urgency=medium

  * Merge from Debian unstable (LP: #1929406). Remaining changes:
    - debian/nfs-common.default: always start idmapd automatically; drop the
      configuration option.
      - debian/nfs-common.default: remove NEED_STATD and related comment.
    - Add 90-gss-free-lucid-sec-context.patch: adjust for changes to the ctx
      argument of the serialize_krb5_ctx() function.
    - Add remove-gssproxy.patch: Drop gssproxy as it does not exist in Ubuntu
    - Fixing nfs-mountd dependency on rpcbind (race condition) by adding
      "rpcbind.socket" to "nfs-mountd.service" as a dependency to avoid race
      conditions:
      - Add systemd-Fix-nfs-mountd-dependency-on-rpcbind.patch
    - truncate_gid*.patch: Backports from upstream to prevent truncating UIDs
      and GIDs over 65536 on certain architectures
    - d/p/fix-start-ordering-1.patch, d/p/fix-start-ordering-2.patch, d/p/fix-
      start-ordering-3.patch, d/nfs-kernel-server.install: Fix systemd service
      start ordering
    - Depend on network-online.target when starting services.
      - d/p/lp1918141-use-network-online-target-01.patch: Declare a
        Wants=network-online.target on all NFS server services.
      - d/p/lp1918141-use-network-online-target-02.patch: Declare a
        After=network-online.target on all NFS server services. Thanks to
        Niklas Edmundsson for helping with the reproducer.
    - d/nfs-common.postinst: always start nfs-utils.service, so the restart in
      the #DEBHELPER# section can do its job if needed
  * Removed patches obsoleted/merged by upstream:
    - Remove hard-coded dep on libtirpc1; already merged upstream, but
      previously un-noted
    - d/nfs-utils_env.sh: alongside RPCSVCGSSDARGS, also export SVCGSSDARGS,
      which is the variable name expected by the rpc-svcgssd systemd service;
      already merged upstream, but previously un-noted
    - d/p/python3.diff obsoleted by
      d/p/Convert-remaining-python-scripts-to-python3.patch
  * d/p/nfsiostat-replace-list-reserved-word.patch: re-instated after
    previous merge removed a patch already merged upstream
  * d/p/01-sm-notify-in-sbin.patch,
    d/p/03-handle-mtab-symlink.patch,
    d/p/17-multiarch-kerberos-paths.patch,
    d/p/unbreak-blkmapd-rpc_pipefs-run.patch:
    reverted unnecessary refresh of patches

[Steve Dodd (anarchetic)]

Bug #1901709 should be fixed by a resync, too. I don't know how many people other than me are mad enough to be running nfs servers in containers ..

[Andreas Hasenack (ahasenack)]

I'll tentatively assign this to me, as I worked on nfs-utils in the somewhat recent past. But don't consider this an exclusive lock ;)

[Andreas Hasenack (ahasenack)]

Starting the work on this

[Andreas Hasenack (ahasenack)]

Working with test packages at https://launchpad.net/~ahasenack/+archive/ubuntu/nfs-utils-merge

[Andreas Hasenack (ahasenack)]

Many bits of delta dropped, nice.

So far, with the ppa packages, I re-tested these to make sure some of the delta drops are not reintroducing their respective bugs:
- bug #1331201: rpc.gssd: ERROR: GSS-API: error in gss_free_lucid_sec_context(): GSS_S_NO_CONTEXT
- bug #1871214: [SRU] nfsd doesn't start if exports depend on mount (exporting via nfs an iscsi mount)

All good. Next I want to test gssproxy integration, which is another delta we had.

[Andreas Hasenack (ahasenack)]

I dropped the delta for bug #1446851, made sure gssproxy was not installed, and I don't see systemd errors or warnings about gssproxy not being there, so all good there too.

[Reza Sherafat (rezas)]

@Andreas, is there a tentative timeline for this to be released in Ubuntu 22.04?

PS. The feature that I'm interested in is .k5identity support in rpc.gssd.

[Andreas Hasenack (ahasenack)]

The next big chunk of work is how to deal with the configuration file change, from /etc/default/nfs-* to /etc/nfs.conf. But I will put up a Merge Proposal next week without that, so it lands in Jammy and gets a wider audience, and work on the upgrade path after that.

[Siddhartha Bedi (sidbedi)]

Thank you for your efforts on this. This is an important upgrade for me. I require the nfs re-export features of knfsd and nsf-utils needs to be > 1.3.5; it has been frozen on 1.3.4. This allows me to use re-exporting with cachefilesd to provide cloud / wan acceleration for on premise file shares.

I was able to utilize your code to backport this to Ubuntu 20.04 to successfully get these features to work.

[Andreas Hasenack (ahasenack)]

> Thank you for your efforts on this.

Thanks, but just to be clear, this is mostly Debian work. I'm just re-applying our specific ubuntu changes on top of that, where I deem still appropriate.

Thanks for testing the packages I built so far. I'll work on getting a PR up for ubuntu shortly, without the migration path yet.

[Steve Dodd (anarchetic)]

I've rebuilt my 'LAN services' container with the packages linked here, and nothing seems to have exploded over the last hour or so. Not sure if that constitutes extensive testing :) As it seems stable I'll leave it running indefinitely to catch any wrinkles..

[Andreas Hasenack (ahasenack)]

Thanks!

Please pay attention to any parameters you were using in /etc/default/nfs-*, as they are not migrated automatically to /etc/nfs.conf, which is the new place. The /etc/default/nfs-* files are completely ignored now.

[Launchpad Janitor (janitor)]

This bug was fixed in the package nfs-utils - 1:2.6.1-1~exp1ubuntu1

---------------
nfs-utils (1:2.6.1-1~exp1ubuntu1) jammy; urgency=medium

  * Merge with Debian experimental (LP: #1878601, #1812280). Remaining
    changes:
    - d/nfs-common.postinst: always start nfs-utils.service, so the restart in
      the #DEBHELPER# section can do its job if needed
  * Dropped:
    - Add remove-gssproxy.patch: Drop gssproxy as it does not exist in Ubuntu
      [Ubuntu has it, but not in main]
    - debian/nfs-common.default: remove NEED_STATD and related comment.
      [/etc/default/nfs-* files are no longer used]
    - debian/nfs-common.default: always start idmapd automatically; drop the
      configuration option.
      [/etc/default/nfs-* files are no longer used]
    - Add 90-gss-free-lucid-sec-context.patch: adjust for changes to the ctx
      argument of the serialize_krb5_ctx() function.
      [Was only needed when libgssglue-dev was used for build, which it
      isn't since nfs-utils 1:1.2.8-7]
    - Fixing nfs-mountd dependency on rpcbind (race condition) by adding
      "rpcbind.socket" to "nfs-mountd.service" as a dependency to avoid race
      conditions:
      - Add systemd-Fix-nfs-mountd-dependency-on-rpcbind.patch
      [Fixed upstream]
    - truncate_gid*.patch: Backports from upstream to prevent truncating UIDs
      and GIDs over 65536 on certain architectures
      [Fixed upstream]
    - d/p/fix-start-ordering-1.patch, d/p/fix-start-ordering-2.patch, d/p/fix-
      start-ordering-3.patch, d/nfs-kernel-server.install: Fix systemd service
      start ordering
      [Fixed upstream. Patch 3 we don't need anymore because now the pkg
      is built with --with-systemd]
    - Depend on network-online.target when starting services.
      - d/p/lp1918141-use-network-online-target-01.patch: Declare a
        Wants=network-online.target on all NFS server services.
      - d/p/lp1918141-use-network-online-target-02.patch: Declare a
        After=network-online.target on all NFS server services. Thanks to
        Niklas Edmundsson for helping with the reproducer.
        [Fixed upstream]
  * Added:
    - d/control: don't provide libnfsidmap2 in libnfsidmap1
    - d/NEWS: explain some of the major changes in 2.6.x
    - Don't install the regex module, as it's built by
      src:libnfsidmap-regex which is in Universe (MIR: #1960824)
      + d/control: don't conflict/break/etc with libnfsidmap-regex
      + d/libnfsidmap1.install: don't install regex.so
      + d/not-installed: mark files we knowingly don't include in the
        packaging
      + d/p/remove-regex-from-docs.patch: remove the regex section from
        the idmapd.conf(5) manpage, as we are not building that plugin in
        this package
    - Update README files:
      + d/README.Ubuntu: explains some of the packaging decisions
      + d/README.Debian.nfsv4: removed as the content is out of date
      + d/nfs-common.docs: install README.Ubuntu instead of
        README.Debian.nfsv4
      + d/libnfsidmap1.docs: install README.Ubuntu

 -- Andreas Hasenack <email address hidden> Wed, 16 Feb 2022 20:20:26 +0000

[3vi1 (launchpad-net-eternaldusk)]

Errr... doesn't this update completely break sssd, since the new package depends on libnfsidmap1 whereas sssd depends on libnfsidmap2? I can't seem to have both sssd and nfs-common installed in jammy since this update.

[Andreas Hasenack (ahasenack)]

Sssd mas rebuilt, but it hasn't migrated yet. I don't know why Britney
migrated nfs-utils without the rebuilt sssd. In any case, this should be
temporary.

On Wed, Feb 23, 2022, 21:35 3vi1 <email address hidden> wrote:

> Errr... doesn't this update completely break sssd, since the new package
> depends on libnfsidmap1 whereas sssd depends on libnfsidmap2? I can't
> seem to have both sssd and nfs-common installed in jammy since this
> update.
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1878601
>
> Title:
> Merge nfs-utils from Debian experimental for 22.04 - version in Ubuntu
> is *very* old
>
> Status in Ubuntu Server Guide:
> Triaged
> Status in Release Notes for Ubuntu:
> New
> Status in nfs-utils package in Ubuntu:
> Fix Released
> Status in nfs-utils package in Debian:
> Fix Released
>
> Bug description:
> nfs-utils has fallen FAR behind upstream, in both Debian and Ubuntu;
> we've had 1.3.4 since Bionic:
>
> $ rmadison -u debian nfs-utils
> nfs-utils | 1:1.2.8-9 | oldoldstable | source
> nfs-utils | 1:1.3.4-2.1 | oldstable | source
> nfs-utils | 1:1.3.4-2.5 | stable | source
> nfs-utils | 1:1.3.4-3 | testing | source
> nfs-utils | 1:1.3.4-3 | unstable | source
> nfs-utils | 1:1.3.4-3 | unstable-debug | source
> $ rmadison nfs-utils
> nfs-utils | 1:1.2.5-3ubuntu3 | precise | source
> nfs-utils | 1:1.2.5-3ubuntu3.2 | precise-updates | source
> nfs-utils | 1:1.2.8-6ubuntu1 | trusty | source
> nfs-utils | 1:1.2.8-6ubuntu1.2 | trusty-updates | source
> nfs-utils | 1:1.2.8-9ubuntu12 | xenial | source
> nfs-utils | 1:1.2.8-9ubuntu12.2 | xenial-updates | source
> nfs-utils | 1:1.3.4-2.1ubuntu5 | bionic | source
> nfs-utils | 1:1.3.4-2.1ubuntu5.2 | bionic-updates | source
> nfs-utils | 1:1.3.4-2.3ubuntu3 | disco | source
> nfs-utils | 1:1.3.4-2.5ubuntu2 | eoan | source
> nfs-utils | 1:1.3.4-2.5ubuntu3 | focal | source
> nfs-utils | 1:1.3.4-2.5ubuntu3 | groovy | source
>
> Version 1.3.4 is from 2016, so we are 4 years behind upstream.
>
> https://kernel.org/pub/linux/utils/nfs-utils/
>
> Debian has an 'upstream' branch:
> https://salsa.debian.org/kernel-team/nfs-utils/-/tree/upstream
>
> as well as a WIP MR:
> https://salsa.debian.org/kernel-team/nfs-utils/-/merge_requests/1
>
> and a newer WIP MR:
> https://salsa.debian.org/kernel-team/nfs-utils/-/merge_requests/3
>
>
> Someone should get nfs-utils updated in Debian and Ubuntu.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/serverguide/+bug/1878601/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=serverguide; status=Triaged; importance=Undecided;
> <email address hidden>;
> Launchpad-Bug: product=ubuntu-release-notes; status=New;
> importance=Undecided; <email address hidden>;
> Launchpad-Bug: distribution=ubuntu; sourcepackage=nfs-utils;
> component=main; milestone=ubuntu-22.01; status=Fix Released;
> importa...

[Steve Langasek (vorlon)]

On Thu, Feb 24, 2022 at 12:46:28AM -0000, Andreas Hasenack wrote:
> Sssd mas rebuilt, but it hasn't migrated yet. I don't know why Britney
> migrated nfs-utils without the rebuilt sssd.

Because co-installability of sssd and nfs is not a constraint. Packages are
allowed to coexist in the release pocket that conflict and are not
co-installable.

But yes, this will all settle out when sssd migrates.

[Andreas Hasenack (ahasenack)]

I updated the release notes and server guide.