Comment 10 for bug 1778219

Revision history for this message
Balint Reczey (rbalint) wrote :

The hang can also occur when unattended-upgrades is started by apt's timer (u-u is modified to help hitting it at the right time):

$ cat reproduce-apt-ran-uu.sh
#!/bin/sh

set -e

apt update
apt upgrade -y
echo 'Unattended-Upgrade::Allowed-Origins:: "${distro_id}:${distro_codename}-updates";' > /etc/apt/apt.conf.d/51unattended-upgrades-updates-too
case $(lsb_release -c -s) in
    bionic)
        apt install --allow-downgrades -y apport=2.20.9-0ubuntu7 snapd=2.32.5+18.04
        ;;
esac

grep -q 'sleep' /usr/bin/unattended-upgrade || sed -i 's/import atexit/import time\nimport atexit/;s/res = cache\.commit/time.sleep(5)\n res = cache.commit/' /usr/bin/unattended-upgrade

rm -f /var/lib/apt/periodic/u*
apt upgrade -d -y
service apt-daily-upgrade start &
sleep 7
reboot

@juliank In https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1690980/comments/50 you suggested inhibiting shutdown for the commit operations only in u-u, but IMO that would still leave the race condition open where the user can start shutdown between commits thus possibly hanging the system with the issue reported in this bug.

One option (1) would be u-u starting the inhibition at the beginning of its run via dbus, the other (2) would be apt.systemd.daily calling u-u with the systemd-inhibit wrapper.
2 would be cleaner and u-u would not have to grow dependency on Python3 dbus modules.

Apt.systemd.daily calling u-u with systemd-inhibit would not back shutdown for long on average since it is already unlikely to hit an upgrade with a shutdown, but in the worst case installing all bionic security updates can take 10-s of minutes. Again, this is a highly unlikely scenario, but if users are hitting that u-u could still get support for monitoring inhibited shutdown requests on dbus and gracefully stop without applying all updates.

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/u/unattended-upgrades/20181010_011153_2b8c7@/log.gz