unattended-upgrade-shutdown hangs when /var is a separate filesystem
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
unattended-upgrades (Debian) |
Fix Released
|
Unknown
|
|||
unattended-upgrades (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Xenial |
Fix Released
|
High
|
Unassigned | ||
Yakkety |
Fix Released
|
High
|
Unassigned | ||
Zesty |
Fix Released
|
High
|
Unassigned |
Bug Description
[SRU justification]
This fix is needed to make sure that the system does not hang on shutdown when /var is a speparate file system
[Impact]
System can hang up to 10 minutes if not fixed.
[Fix]
Change the systemd unit's ExecStart to an ExecStop so the unit is correctly sequenced.
Change WantedBy to multi-user.target. This requires working around Debian Bug #797108 which causes the new unit not to be enabled.
Remove the unneeded override_
Add Default-Start levels to the SysV initscript
Improve DEP8 testing
[Test Case]
In a VM with /var separated from the / file system, shutdown the VM. It will hang for 10 minutes
[Regression]
Upgrade has been tested on Xenial, Yakkety, Zesty. do-release-upgrade has been tested from Trusty to Xenial. All behave as expected.
A change of behavior may occur now that the systemd unit is correctly enabled, which would make functionalities like InstallOnShutdown to work as expected whereas it could have been broken previously. This cannot be considered as a regression as it is expected behavior.
Shutdown may be slowed down as it now correctly depends on /var and /boot to be available so the unit will run earlier than previously.
[Original description of the problem]
The systemd unit file unattended-
process during shutdown. This unit file is running together with all filesystem
unmount services.
The unattended-upgrades service checks if the lockfile for unattended-upgrade
(in /var/run) exists, and if it does, there is an unattended-upgrade in progress
and the service will wait until it finishes (and therefore automatically wait at
shutdown).
However, if /var is a separate filesystem, it will get unmounted even though /var/run
is a tmpfs that's still mounted on top of the /var/run directory in the /var filesystem.
The unattended-upgrade script will fail to find lockfile, sleeps for 5 seconds, and
tries to check the lockfile again. After 10 minutes (the default timeout), it will finally
exit and the system will continue shutdown.
The problem is the error handling in /usr/share/
where it tries to lock itself:
while True:
res = apt_pkg.
# exit here if there is no lock
if res > 0:
break
The function apt_pkg.get_lock() either returns a file descriptor, or -1 on an error.
File descriptors are just C file descriptors, so they are always positive integers.
The code should check the result to be negative, not positive. I have attached a patch
to reverse the logic.
Additional information:
1)
Description: Ubuntu 16.04.1 LTS
Release: 16.04
2)
unattended-
Installed: 0.90ubuntu0.3
Candidate: 0.90ubuntu0.3
Version table:
*** 0.90ubuntu0.3 500
500 http://
500 http://
100 /var/lib/
0.90 500
500 http://
500 http://
3)
Fast reboot
4)
Very slow reboot (after a 10 minutes timeout)
tags: | added: patch |
Changed in unattended-upgrades (Ubuntu): | |
importance: | Undecided → High |
Changed in unattended-upgrades (Ubuntu): | |
assignee: | nobody → Louis Bouchard (louis-bouchard) |
Changed in unattended-upgrades (Ubuntu Xenial): | |
assignee: | nobody → Louis Bouchard (louis-bouchard) |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
assignee: | nobody → Louis Bouchard (louis-bouchard) |
Changed in unattended-upgrades (Ubuntu Xenial): | |
importance: | Undecided → High |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
importance: | Undecided → High |
Changed in unattended-upgrades (Ubuntu Xenial): | |
status: | New → Confirmed |
Changed in unattended-upgrades (Debian): | |
status: | Unknown → New |
Changed in unattended-upgrades (Ubuntu): | |
status: | Confirmed → In Progress |
description: | updated |
Changed in unattended-upgrades (Ubuntu Xenial): | |
status: | Confirmed → In Progress |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
status: | Confirmed → In Progress |
Changed in unattended-upgrades (Ubuntu): | |
status: | Fix Released → In Progress |
Changed in unattended-upgrades (Ubuntu Xenial): | |
status: | Fix Committed → Triaged |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
status: | Fix Committed → Triaged |
tags: | added: sts-sru-needed |
description: | updated |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
status: | Triaged → In Progress |
Changed in unattended-upgrades (Ubuntu Xenial): | |
status: | Triaged → In Progress |
Changed in unattended-upgrades (Ubuntu): | |
assignee: | Louis Bouchard (louis) → nobody |
tags: |
added: verification-done removed: sts-sru-needed verification-needed |
Changed in unattended-upgrades (Debian): | |
status: | New → Fix Released |
Changed in unattended-upgrades (Ubuntu Xenial): | |
assignee: | Louis Bouchard (louis) → nobody |
Changed in unattended-upgrades (Ubuntu Yakkety): | |
assignee: | Louis Bouchard (louis) → nobody |
Changed in unattended-upgrades (Ubuntu Zesty): | |
assignee: | Louis Bouchard (louis) → nobody |
I have also tried to let the systemd unit file for this service WantedBy the umount.target. upgrade. service reverse depends on the shutdown.target.
According to the systemd documentation that should make it run before the filesystems
are unmounted, but apparently that doesn't work. systemctl list-dependencies still shows
that the unattended-