useradd --extrausers --groups tries to lock /etc/group

[Valentin David]

[ Impact ]

On Ubuntu Core 24 calling the command line

useradd --extrausers --groups somegroup somenewuser

... fails with:

useradd: cannot lock /etc/group; try again later.

It worked on 22.04. /etc is not writable. It also fails if somegroup is a group in extrausers.

[ Test Plan ]

Part of the upload is adding an autopkgtest script testing useradd and usermod in the extrausers+readonly-etc case.

In addition, the following commands should be run as root in a fresh container:

```
# Install prerequisites
apt install libnss-extrausers
sed -i -r -e'/^(passwd|group|shadow|gshadow)/ s/$/ extrausers/' /etc/nsswitch.conf # enable extrausers in group, passwd, shadow and gshadow

# Sanity checks of "normal" path
groupadd etcgroup
useradd --groups etcgroup etcuser
id etcuser | grep etcgroup
groupadd etcgroup2
usermod --groups etcgroup2 etcuser
id etcuser | grep etcgroup2
useradd --groups nullgroup etcuser || echo Successfully rejected invalid group

ls /var/lib/extrausers/ # should be empty

# Sanity checks of "extrausers" path in rw context
groupadd --extrausers extragroup
useradd --extrausers --groups extragroup extrauser # currently fails
id extrauser | grep extragroup
useradd --extrausers extrauser2
id extrauser2
usermod --groups extragroup extrauser2
id extrauser2 | grep extragroup

# Sanity checks of "extrausers" path in ro context
mv /etc /etc-rw
mkdir /etc
mount -o bind,ro /etc-rw /etc
groupadd --extrausers extragroup2
useradd --extrausers --groups etcgroup extrauser3
id extrauser4 | grep etcgroup
usermod --groups extragroup2 extrauser3
id extrauser4 | grep extragroup2
```

Furthermore, validation from the Ubuntu Core team that this actually fixes
their use case is required.

[ Where problems could occur ]

Regression potential is in the group validation stage of the `usermod` and
`useradd` tools. Besides the usual risks related to C code, the various failure
scenarios that come to mind are:

* try to add the user to an non-existing local group, which would fail further
  down with a different error message
* actually fail to identify a valid local group
* Fail to either add the user to the system, or the user to the group
* Update the wrong file (/var/lib/extrausers/* vs /etc/*)

[Simon Chopin (schopin)]

Quick repro steps:

❯ lxc launch ubuntu-daily:noble shadow
Creating shadow
Starting shadow
❯ lxc exec shadow bash
root@shadow:~# mv /etc /etc_write
root@shadow:~# mkdir /etc
root@shadow:~# mount -o bind,ro /etc_write /etc
root@shadow:~# useradd --extrausers --groups somegroup somenewuser
useradd: cannot lock /etc/group; try again later.

[Simon Chopin (schopin)]

The issue was introduced with https://github.com/shadow-maint/shadow/pull/237

Basically, the previous group validation was done using glibc's getgrid directly, which was presumably coping well with the RO status of /etc/group, but that poses consistency problems because you could add a local user to a network group. That PR changed this to only check the local /etc/group file contents manually instead.

Sadly, it doesn't cope well with our extrausers feature on multiple levels:
* The manual code fails hard if it can't lock the files
* We presumably have local groups defined in multiple places, which the code doesn't allow for.

A quickfix would be:
* Move the validation to until *after* parsing all of the options
* Revert back to the previous approach to validate groups if in extrausers mode

A more involved fix would be to replace that with an approach that would check both /etc/group and the extrausers equivalent when validating groups, while silently ignoring locking failures.

[Simon Chopin (schopin)]

I opted for the "quickfix" approach, as it has the merits of minimizing the changes to the "normal" codepaths, thus reducing the risks of regressions for non-core users. While the extrausers case is not as correct as it would have been with a full fix, it's consistent with the behaviour in 22.04 and thus should remain compatible with all existing code.

I also apparently fumbled my LP: #XXXXXXX stanza in the oracular changelog so I'll have to update the bug state manually.

[Simon Chopin (schopin)]

I marked Mantic as Wont Fix since this is a bug that primarily affects Ubuntu Core, Mantic users are likely unaffected, so I'd rather spare them an unnecessary update.

[Steve Langasek (vorlon)]

Hello Valentin, or anyone else affected,

Accepted shadow into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shadow/1:4.13+dfsg1-4ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted shadow (1:4.13+dfsg1-4ubuntu3.1) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

bash/unknown (s390x)
buildbot/unknown (s390x)
debvm/unknown (s390x)
dh-sysuser/unknown (s390x)
dnsproxy/unknown (s390x)
gdm3/unknown (s390x)
lxc/unknown (arm64)
mysql-8.0/unknown (arm64)
rancid/unknown (arm64)
samba/unknown (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#shadow

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Marius Vollmer (marius-vollmer-gmail)]

"useradd -D" has started to fail with passwd 1:4.13+dfsg1-4ubuntu3.1:

# useradd -D
Usage: useradd [options] LOGIN
       useradd -D
       useradd -D [options]

Options:
      --badname do not check for bad names
  -b, --base-dir BASE_DIR base directory for the home directory of the
                                new account
...
# echo $?
2

[Simon Chopin (schopin)]

Thanks for testing and catching that regression! I've found and fixed the issue in an oracular upload. Once it clears -proposed I'll amend the noble-proposed version.

[Launchpad Janitor (janitor)]

This bug was fixed in the package shadow - 1:4.13+dfsg1-4ubuntu5

---------------
shadow (1:4.13+dfsg1-4ubuntu5) oracular; urgency=medium

  * d/p/lp2063200/*: amend the patch to fix `useradd -D` breakage
    (LP: #2063200)

 -- Simon Chopin <email address hidden> Mon, 27 May 2024 18:56:51 +0200

[Timo Aaltonen (tjaalton)]

Hello Valentin, or anyone else affected,

Accepted shadow into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/shadow/1:4.13+dfsg1-4ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.