Comment 9 for bug 1654600

Revision history for this message
Erik Mouw (erikzbhj) wrote : Re: [Bug 1654600] Re: unattended-upgrade-shutdown hangs when /var is a separate filesystem

You're right, I thought it was the lockfile for the shutdown helper only.

Somewhere else in this thread I already mentioned that a dependency on the local_fs and remote_fs targets fixes the systemd order:

Before=remote-fs.target local-fs.target

That makes sure the service is run before all file systems are unmounted. If you add the shutdown.target to that rule, systemd somehow thinks it should be run just before shutdown which is obviously too late. I'm not sure that is a systemd bug or a systemd feature, but either way it doesn't do what we want.

Regards,

Erik

--
Erik Mouw

> Op 26 jan. 2017 om 18:00 heeft Louis Bouchard <email address hidden> het volgende geschreven:
>
> Hello,
>
> This logic is correct. apt_pkg.get_lock(options.lock_file) is trying to
> acquire a lockfile (/var/run/unattended-upgrades.lock) which is only
> possible if no unattended-upgrade is already running. If it can acquire
> the lock, it means that NO unattended-upgrade is running, hence shutdown
> may proceed.
>
> while True:
> res = apt_pkg.get_lock(options.lock_file)
> logging.debug("get_lock returned %i" % res)
> # exit here if there is no lock
> if res > 0:
> logging.debug("lock not taken")
> break
> lock_was_taken = True
>
> The problem is that apt_pkg.get_lock(options.lock_file) will also return
> -1 if the path to the lock file doesn't exist, which is the case when
> /var is unmounted, hence /var/run is no longer present (/var/run is a
> symlink to /run).
>
> I'm working on identifying the proper systemd order to make sure that
> /var/run is still accessible.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1654600
>
> Title:
> unattended-upgrade-shutdown hangs when /var is a separate filesystem
>
> Status in unattended-upgrades package in Ubuntu:
> Confirmed
> Status in unattended-upgrades source package in Xenial:
> Confirmed
> Status in unattended-upgrades source package in Yakkety:
> New
>
> Bug description:
> The systemd unit file unattended-upgrades.service is used to stop a running unattended-upgrade
> process during shutdown. This unit file is running together with all filesystem
> unmount services.
>
> The unattended-upgrades service checks if the lockfile for unattended-upgrade
> (in /var/run) exists, and if it does, there is an unattended-upgrade in progress
> and the service will wait until it finishes (and therefore automatically wait at
> shutdown).
>
> However, if /var is a separate filesystem, it will get unmounted even though /var/run
> is a tmpfs that's still mounted on top of the /var/run directory in the /var filesystem.
> The unattended-upgrade script will fail to find lockfile, sleeps for 5 seconds, and
> tries to check the lockfile again. After 10 minutes (the default timeout), it will finally
> exit and the system will continue shutdown.
>
> The problem is the error handling in /usr/share/unattended-upgrades/unattended-upgrade-shutdown
> where it tries to lock itself:
>
> while True:
> res = apt_pkg.get_lock(options.lock_file)
> logging.debug("get_lock returned %i" % res)
> # exit here if there is no lock
> if res > 0:
> logging.debug("lock not taken")
> break
> lock_was_taken = True
>
> The function apt_pkg.get_lock() either returns a file descriptor, or -1 on an error.
> File descriptors are just C file descriptors, so they are always positive integers.
> The code should check the result to be negative, not positive. I have attached a patch
> to reverse the logic.
>
> Additional information:
>
> 1)
> Description: Ubuntu 16.04.1 LTS
> Release: 16.04
>
> 2)
> unattended-upgrades:
> Installed: 0.90ubuntu0.3
> Candidate: 0.90ubuntu0.3
> Version table:
> *** 0.90ubuntu0.3 500
> 500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
> 500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages
> 100 /var/lib/dpkg/status
> 0.90 500
> 500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
> 500 http://nl.archive.ubuntu.com/ubuntu xenial/main i386 Packages
> 3)
> Fast reboot
> 4)
> Very slow reboot (after a 10 minutes timeout)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1654600/+subscriptions