systemd journald failed unmounting var

Bug #1788048 reported by Adauto Serpa on 2018-08-20
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemd
Fix Released
Unknown
systemd (Ubuntu)
Status tracked in Eoan
Bionic
Medium
Unassigned
Cosmic
Medium
Unassigned
Disco
Medium
Unassigned
Eoan
Medium
Unassigned

Bug Description

[impact]

on a system with /var and/or /var/log mounted separately from /, shutdown can't unmount /var (or /var/log) cleanly because systemd-journald is using it.

[test case]

install a system with /var mounted separately from /

shutdown, and notice:

[FAILED] Failed unmounting /var.

[regression potential]

TBD

[other info]

original description:

--

When the system is shut down or restarted the systemd-journal does not disassemble the disks correctly.

On this thread https://unix.stackexchange.com/questions/378678/why-do-i-get-the-error-failed-unmounting-var-during-shutdown

I checked what editing /etc/systemd/journald.conf
to change the Storage = line to Storage = volatile the problem is solved, but some of the logs on the shutdown are lost.

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: systemd 237-3ubuntu10.3
ProcVersionSignature: Ubuntu 4.15.0-32.35-generic 4.15.18
Uname: Linux 4.15.0-32-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.2
Architecture: amd64
Date: Mon Aug 20 18:15:07 2018
ExecutablePath: /lib/systemd/systemd-journald
InstallationDate: Installed on 2018-08-20 (0 days ago)
InstallationMedia: Ubuntu-Server 18.04.1 LTS "Bionic Beaver" - Release amd64 (20180725)
Lsusb:
 Bus 001 Device 002: ID 80ee:0021 VirtualBox USB Tablet
 Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
MachineType: innotek GmbH VirtualBox
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-32-generic root=UUID=65cb761b-4881-4031-9113-e6bb6a1437b4 ro
SourcePackage: systemd
SystemdDelta:
 [EXTENDED] /lib/systemd/system/rc-local.service → /lib/systemd/system/rc-local.service.d/debian.conf
 [EXTENDED] /lib/systemd/system/user@.service → /lib/systemd/system/user@.service.d/timeout.conf

 2 overridden configuration files found.
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 12/01/2006
dmi.bios.vendor: innotek GmbH
dmi.bios.version: VirtualBox
dmi.board.name: VirtualBox
dmi.board.vendor: Oracle Corporation
dmi.board.version: 1.2
dmi.chassis.type: 1
dmi.chassis.vendor: Oracle Corporation
dmi.modalias: dmi:bvninnotekGmbH:bvrVirtualBox:bd12/01/2006:svninnotekGmbH:pnVirtualBox:pvr1.2:rvnOracleCorporation:rnVirtualBox:rvr1.2:cvnOracleCorporation:ct1:cvr:
dmi.product.family: Virtual Machine
dmi.product.name: VirtualBox
dmi.product.version: 1.2
dmi.sys.vendor: innotek GmbH
mtime.conffile..etc.systemd.journald.conf: 2018-08-20T18:00:58.586165

Adauto Serpa (adautoserpa) wrote :
Adauto Serpa (adautoserpa) wrote :

One more discussion about systemctl does not work correctly with the separate / var partition
https://bbs.archlinux.org/viewtopic.php?pid=1204644#p1204644

Sorry English !

Adauto Serpa (adautoserpa) wrote :

I ended up using another solution, just do not know if it is the most correct.
Edit
sudo vim /lib/systemd/system/systemd-journald.service

Add the line "RequiresMountsFor=/var/log/journal" as below

[Unit]
...
RequiresMountsFor=/var/log/journal

systemctl daemon-reload
systemctl restart systemd-journald
# You'll now see the new setting listed at the bottom of:
systemctl cat systemd-journald.service

Changed in systemd (Ubuntu):
status: New → Confirmed
summary: - failed unmounting var
+ systemd journald failed unmounting var
Changed in systemd (Ubuntu):
status: Confirmed → New
Adauto Serpa (adautoserpa) wrote :

Guys, I think it's a critical problem!
All because I created an exclusive disk for / var

C de-Avillez (hggdh2) wrote :

Updated with upstream bug #; this seems to be an on-going concern, and depends on some not-yet-available interfaces between kernel and systemd.

Marking Triaged/Medium. There are some bypasses discussed in the upstream bug, but not all possible effects have already been discussed (for example, setting 'storage=volatile' completely loses the journald logs at shutdown).

Changed in systemd (Ubuntu):
importance: Undecided → Medium
status: New → Triaged
Adauto Serpa (adautoserpa) wrote :

C de-Avillez,

As I mentioned there is a solution I described in comment #3

Adding just a dependency before dismounting to /var/log/journal

Thank you for your attention!

Changed in systemd:
status: Unknown → New
Changed in systemd:
status: New → Fix Released
Dan Streetman (ddstreet) wrote :

Just to clarify, this does not hang shutdown/reboot for anyone, right? It only fails to cleanly unmount /var, right?

description: updated
Adauto Serpa (adautoserpa) wrote :

@Dan Streetman

That's right, this problem does not affect the reboot or shutdown of the operating system.
My concern is the loss of data due to improper unmount.

Dan Streetman (ddstreet) on 2019-05-15
Changed in systemd (Ubuntu Eoan):
status: Triaged → In Progress
assignee: nobody → Dan Streetman (ddstreet)
Dan Streetman (ddstreet) on 2019-05-15
tags: added: ddstreet-next
Changed in systemd (Ubuntu Disco):
status: New → In Progress
Changed in systemd (Ubuntu Cosmic):
status: New → In Progress
Changed in systemd (Ubuntu Bionic):
status: New → In Progress
Changed in systemd (Ubuntu Disco):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Cosmic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Disco):
importance: Undecided → Medium
Changed in systemd (Ubuntu Bionic):
importance: Undecided → Medium
Changed in systemd (Ubuntu Cosmic):
importance: Undecided → Medium
Dan Streetman (ddstreet) wrote :

It seems like the upstream commits to fix this may have made journalctl --flush flaky and/or introduced other regressions:
https://github.com/systemd/systemd/pull/12230#issuecomment-491593897

Additionally, the fix upstream is not a trivial backport and has the potential to cause even more problems for SRU releases.
https://github.com/systemd/systemd/pull/12230/commits

As this bug is rather limited in scope (only systems with /var or /var/log mounted separately from /) and the only potential issue I see is /var isn't cleanly unmounted, it seems like backporting the upstream fix may bring more problems than it solves.

Possibly there is some simpler workaround we could add for SRU releases, but it would take some more investigation. I'm removing myself as bug owner here, so possibly someone else can investigate, in case I'm not able to get back to this.

tags: removed: ddstreet-next
Changed in systemd (Ubuntu Bionic):
status: In Progress → New
Changed in systemd (Ubuntu Cosmic):
status: In Progress → New
Changed in systemd (Ubuntu Disco):
status: In Progress → New
Changed in systemd (Ubuntu Eoan):
status: In Progress → New
Changed in systemd (Ubuntu Bionic):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Cosmic):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Disco):
assignee: Dan Streetman (ddstreet) → nobody
Changed in systemd (Ubuntu Eoan):
assignee: Dan Streetman (ddstreet) → nobody
Dimitri John Ledkov (xnox) wrote :

Does installing finalrd package help?

It should pivot off root on shutdown, and unmount separate /var and /var/log cleanly.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.