Prompt error message "Failed to unmount /oldroot" when shutdown or reboot

Bug #1928200 reported by Bin Li
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OEM Priority Project
Fix Released
High
Bin Li
systemd (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
High
Dan Streetman

Bug Description

[impact]

root fs is not cleanly unmounted on shutdown

[test case]

create a file in /etc/binfmt.d/TestFormat.conf with the following content:

:TestFormat:M::XX::/bin/true:F

reboot the system, and then shutdown or reboot again, and watch the serial console for the error message:

sd-umount[1334]: Failed to unmount /oldroot: Device or resource busy

[regression potential]

any regression would likely cause problems on shutdown and/or reboot, or may cause problems with unmounted root fs on shutdown/reboot

[scope]

this is needed only for f

this is fixed upstream with PR 15566 which is included in v246, so this is fixed already in g and later

this isn't reproducable on b

[original description]

On ubuntu 20.04, with latest kernel and systemd, it will show error on ThinkPad X1.

sd-umount[1334]: Failed to unmount /oldroot: Device or resource busy

systemd 245.4-4ubuntu3.6
kernel: 5.8.0-53

CVE References

Revision history for this message
Bin Li (binli) wrote :
Revision history for this message
Bin Li (binli) wrote :
tags: added: oem-priority originate-from-1927884 sutton
Revision history for this message
Bin Li (binli) wrote :

Tried to back-port 4 patches from #2, it didn't work for me.

Dan Streetman (ddstreet)
description: updated
description: updated
Changed in systemd (Ubuntu):
status: New → Fix Released
Changed in systemd (Ubuntu Focal):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Dan Streetman (ddstreet)
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Bin, or anyone else affected,

Accepted systemd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/245.4-4ubuntu3.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Bin Li (binli) wrote :

Checked the proposed channel, the 3.8 was not ready yet, will check it later.

Changed in oem-priority:
assignee: nobody → Bin Li (binli)
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Bin Li (binli) wrote :

Upgraded to systemd 245.4-4ubuntu3.8 on ThinkPad P17, but this issue is still exist.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (systemd/245.4-4ubuntu3.8)

All autopkgtests for the newly accepted systemd (245.4-4ubuntu3.8) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

asterisk/1:16.2.1~dfsg-2ubuntu1 (armhf)
gvfs/1.44.1-1ubuntu1 (amd64)
linux-oem-5.6/5.6.0-1057.61 (amd64)
munin/2.0.56-1ubuntu1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Dan Streetman (ddstreet) wrote :

ubuntu@lp1928200-f:~$ cat /etc/binfmt.d/TestFormat.conf
:TestFormat:M::XX::/bin/true:F
ubuntu@lp1928200-f:~$ dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.6 amd64 system and service manager

reboot to pick up change...

reboot again to show issue...

[ OK ] Reached target Reboot.
[ 35.164202] sd-umoun[1548]: Failed to unmount /oldroot: Device or resource busy
[ 35.174070] shutdown[1]: Failed to finalize file systems, ignoring
[ 35.219215] reboot: Restarting system

ubuntu@lp1928200-f:~$ dpkg -l systemd|grep systemd
ii systemd 245.4-4ubuntu3.8 amd64 system and service manager

reboot to pick up change...

reboot again to check issue...

[ OK ] Reached target Reboot.
[ 22.494535] reboot: Restarting system

tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Dan Streetman (ddstreet) wrote :

> Upgraded to systemd 245.4-4ubuntu3.8 on ThinkPad P17, but this issue is still exist.

you may be seeing some other problem, you should open a new bug

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemd - 245.4-4ubuntu3.10

---------------
systemd (245.4-4ubuntu3.10) focal-security; urgency=medium

  * SECURITY UPDATE: DoS via DHCP FORCERENEW
    - debian/patches/CVE-2020-13529.patch: tentatively ignore FORCERENEW
      command in src/libsystemd-network/sd-dhcp-client.c.
    - CVE-2020-13529
  * SECURITY UPDATE: denial of service via stack exhaustion
    - debian/patches/CVE-2021-33910.patch: do not use strdupa() on a path
      in src/basic/unit-name.c.
    - CVE-2021-33910

 -- Marc Deslauriers <email address hidden> Tue, 20 Jul 2021 07:39:51 -0400

Changed in systemd (Ubuntu Focal):
status: Fix Committed → Fix Released
Bin Li (binli)
Changed in oem-priority:
status: Confirmed → Fix Released
Revision history for this message
Daniel (daniel-marynicz) wrote (last edit ):

I still have this problem, but I have solved it.

I create files

file: /etc/systemd/system/kill-plymouthd.service
[Unit]
Description=kill plymouthd
After=umount.target
DefaultDependencies=false

[Service] Type=oneshot
ExecStart=/usr/bin/pkill plymouthd

[Install] WantedBy=final.target

file: /etc/systemd/system/rm-nvidia.service
[Unit]
Description=rm nvidia modules
After=umount.target
DefaultDependencies=false

[Service] Type=oneshot
ExecStart=/usr/sbin/rmmod nvidia_uvm
ExecStart=/usr/sbin/rmmod nvidia_drm
ExecStart=/usr/sbin/rmmod nvidia_modeset
ExecStart=/usr/sbin/rmmod nvidia
ExecStart=/usr/sbin/rmmod i2c_nvidia_gpu

[Install]
WantedBy=final.target

execute:
sudo systemctl daemon-reload
sudo systemctl enable rm-nvidia.service
sudo systemctl enable kill-plymouthd.service

for debug this problem i installed finalrd ane created executable

/etc/finalrd/sleep.finalrd

#!/bin/sh

set -x
if [ "$1" = "setup" ] then
    . /usr/share/initramfs-tools/hook-functions
    copy_exec /usr/bin/sleep
    copy_exec /usr/bin/mount

    exit 0
fi

#your debug script
mount
sleep 30

i have second system with ubuntu and in this second system i do not have nvidia drivers and i only created kill-plymouthd.service and this also work.

 and i did know why plymouthd is not killed befor umount root fs..

Revision history for this message
Bin Li (binli) wrote :

@daniel,

 Thanks for the solution, and it sounds good, could you help make merge request to finalrd to ask the maintainer's idea?

 From my side, currently finalrd used by mdadm, I would like to remove the dependency cause mdadm.

 In systemd/mdadm.shutdown from mdadm, it also do "mdadm --wait-clean --scan".

$ cat ./debian/mdadm.finalrd
#!/bin/sh
# SPDX-License-Identifier: GPL-3.0-only
set -e

if [ "$1" = "setup" ]
then
    . /usr/share/initramfs-tools/hook-functions
    copy_exec /sbin/mdadm
    copy_exec /sbin/mdmon

    exit 0
fi

mdmon --all --takeover
mdadm -vv --wait-clean --scan
mdadm -vv --stop --scan

 Thanks!

Revision history for this message
Bin Li (binli) wrote (last edit ):

@daniel,

 I reported this issue in another ticket. Let's focus on below links. Thanks!

https://bugs.launchpad.net/ubuntu/+source/finalrd/+bug/1940514

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.