Bind mount to NFS mount fails on Ubuntu 22.04
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
systemd |
Unknown
|
Unknown
|
|||
systemd (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Jammy |
Fix Released
|
Medium
|
Unassigned | ||
Kinetic |
Fix Released
|
Medium
|
Unassigned |
Bug Description
[Impact]
Some NFS setups that have worked on previous versions of Ubuntu are now broken. Specifically, when creating a bind mount, systemd will fail if the directory already exists on NFS.
[Test Plan]
This test plan requires an NFS server to be in place. The exact setup is not important, but for the purposes of this test plan we will assume the server exports /data, which contains a directory called home.
* Make sure nfs-common is installed:
$ apt install nfs-common -y
* Check the NFS server exports using showmount:
$ showmount -e $NFS_SERVER_IP
Export list for $NFS_SERVER_IP:
/data *
* Add the appropriate entries to /etc/fstab. In this example, we will make /home2 a bind mount to /data/home:
# /etc/fstab
$NFS_SERVER_
/data/home /home2 none bind,_netdev,
* Check systemctl status home2.mount. On an affected system, we should see a permission denied error:
$ systemctl status home2.mount
x home2.mount - /home2
Loaded: loaded (/etc/fstab; generated)
Active: failed (Result: resources)
Where: /home2
What: /data/home
Docs: man:fstab(5)
systemd[1]: home2.mount: Failed to make bind mount source '/data/home': Permission denied
systemd[1]: home2.mount: Failed to run 'mount' task: Permission denied
systemd[1]: home2.mount: Failed with result 'resources'.
systemd[1]: Failed to mount /home2.
* On a patched system, we expect /home2 to be mounted successfully.
[Where problems could occur]
The patch makes it so that systemd will not fail bind mounts immediately if creating the directory fails. Thus, any regressions would be seen during bind mounts with systemd. N.B. that this patch restores previous behavior in systemd.
[Original Description]
I have the following setup:
- /data is an NFS mount.
- /home is a bind mount to /data/home.
Configured in /etc/fstab with these lines:
$FILE_SERVER:/data /data nfs defaults 0 0
/data/home /home none bind,_netdev,
This has worked with the same configuration for me on at least Ubuntu 18.04 and Ubuntu 20.04, but on Ubuntu 22.04 the mount of /home fails when attempted by systemd.
So not only is /home not mounted after boot, but also when I run "sudo systemctl start home.mount", it fails.
The journal entries are this:
sudo[1316]: wendler : TTY=pts/0 ; PWD=/home/wendler ; USER=root ; COMMAND=
sudo[1316]: pam_unix(
systemd[1]: home.mount: Directory /home to mount over is not empty, mounting anyway.
systemd[1]: home.mount: Failed to make bind mount source '/data/home': Permission denied
systemd[1]: home.mount: Failed to run 'mount' task: Permission denied
systemd[1]: home.mount: Failed with result 'resources'.
systemd[1]: Failed to mount /home.
However, when I run "sudo mount /home" it works.
Now the weird thing is that after I have mounted /home manually once and unmounted it again, then "sudo systemctl start home.mount" suddenly also works! But of course only until the next reboot.
And even if I just do "ls /data" once (either as root or as my user), it also makes "sudo systemctl start home.mount" start working!
To be fully clear:
- Booting the system (/home fails to mount)
- sudo systemctl start home.mount # fails
- sudo ls /data # shows correct output
- sudo systemctl start home.mount # works!
Additional information:
The NFS export for /data has root_squash set and if I remove this option, the bind mount of /home works as it should. However, both /data and /data/home have mode rwxr-xr-x, so root is able to enter and read these directories even despite root_squash.
It is not a network or mount-order problem. Not only is the bind mount to /home correctly attempted after /data is mounted during boot, remember that the bind mount also fails when I try it with "sudo systemctl start home.mount" minutes afterwards.
Versions (system is a recent installation with all updates applied):
- Ubuntu 22.04
- Linux 5.15.0-40-generic
- systemd 249.11-0ubuntu3.3
- util-linux 2.37.2-4ubuntu3
This appears on both Ubuntu 22.04 machines that I have (a hardware machine and a VM).
Related branches
- Lukas Märdian: Approve
-
Diff: 105 lines (+72/-1)4 files modifieddebian/changelog (+6/-1)
debian/patches/lp1979952-Revert-core-mount-fail-early-if-directory-cannot-be-creat.patch (+33/-0)
debian/patches/lp1979952-core-mount-downgrade-log-level-about-several-mkdir-failur.patch (+31/-0)
debian/patches/series (+2/-0)
Changed in systemd (Ubuntu): | |
status: | New → Triaged |
importance: | Undecided → Medium |
tags: | added: fr-2591 |
tags: | removed: fr-2591 rls-jj-incoming rls-kk-incoming |
description: | updated |
Changed in systemd (Ubuntu Jammy): | |
importance: | Undecided → Medium |
tags: | added: foundations-todo |
tags: | removed: foundations-todo |
Somebody else has reported this upstream (https:/ /github. com/systemd/ systemd/ issues/ 24120) and a fix is being developed as https:/ /github. com/systemd/ systemd/ pull/24122.