Unnattended upgrades take a long time due to excessive running of "apt-check"

[Alistair Buxton]

The unattended upgrade services runs "apt-check" to update the MOTD where a summary of available updates is displayed. It does this after every package that is installed. Unfortunately, apt-check is extremely expensive - it takes about 15 seconds to run.

If you have a computer that is infrequently used, you may have as many as 300 available updates. Unattended upgrade will therefore run apt-check 300 times. This takes 1 hour 15 minutes, during which time no packages are actually installed.

Suppose you turn on your computer once a week to do something like pay some bills. It takes 5 minutes. Then you have to wait 2 hours before you can turn it off. Every time.

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: unattended-upgrades 2.8ubuntu1
ProcVersionSignature: Ubuntu 5.15.0-71.78-generic 5.15.92
Uname: Linux 5.15.0-71-generic x86_64
ApportVersion: 2.20.11-0ubuntu82.4
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: Unity:Unity7:ubuntu
Date: Wed May 15 00:17:47 2024
InstallationDate: Installed on 2013-12-22 (3796 days ago)
InstallationMedia: Ubuntu 13.10 "Saucy Salamander" - Release amd64 (20131016.1)
PackageArchitecture: all
RebootRequiredPkgs: Error: path contained symlinks.
SourcePackage: unattended-upgrades
UpgradeStatus: Upgraded to jammy on 2023-04-16 (394 days ago)
modified.conffile..etc.apt.apt.conf.d.10periodic:
 APT::Periodic::Update-Package-Lists "1";
 APT::Periodic::Download-Upgradeable-Packages "1";
 APT::Periodic::AutocleanInterval "0";
mtime.conffile..etc.apt.apt.conf.d.10periodic: 2017-11-17T12:55:49.319695

[Alistair Buxton (a-j-buxton)]

For example look at .var.log.apt.history.log.txt

Many packages take less than a second to install. But in between each one is a 15 to 20 second delay. That delay is the time during which apt-check is running.

You can watch this happening by running "watch systemctl status apt-daily-upgrade" and looking at the processes in the cgroup:

```
~$ systemctl status apt-daily-upgrade
● apt-daily-upgrade.service - Daily apt upgrade and clean activities
     Loaded: loaded (/lib/systemd/system/apt-daily-upgrade.service; static)
     Active: activating (start) since Tue 2024-05-14 23:01:03 BST; 1h 29min ago
TriggeredBy: ● apt-daily-upgrade.timer
       Docs: man:apt(8)
   Main PID: 29655 (apt.systemd.dai)
      Tasks: 11 (limit: 9299)
     Memory: 1.6G
        CPU: 1h 12min 57.651s
     CGroup: /system.slice/apt-daily-upgrade.service
             ├─ 29655 /bin/sh /usr/lib/apt/apt.systemd.daily install
             ├─ 29659 /bin/sh /usr/lib/apt/apt.systemd.daily lock_is_held install
             ├─ 29688 /usr/bin/python3 /usr/bin/unattended-upgrade
             ├─181903 /usr/bin/python3 /usr/bin/unattended-upgrade
             ├─181970 /usr/bin/python3 /usr/bin/unattended-upgrade
             ├─181978 sh -c "if [ -d /var/lib/update-notifier ]; then touch /var/lib/update-notifier/dpkg-run-stamp; fi; /usr/lib/update-notifier/update-motd-updates-available 2>/dev/null || true"
             ├─181980 /bin/sh -e /usr/lib/update-notifier/update-motd-updates-available
             ├─181994 /usr/bin/python3 /usr/lib/update-notifier/apt-check --human-readable
             └─182339 /usr/bin/python3 -Es /usr/bin/lsb_release -c -s
```

Then manually running "time apt-check"

```
~$ time /usr/lib/update-notifier/apt-check --human-readable
Expanded Security Maintenance for Applications is not enabled.

269 updates can be applied immediately.
45 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

33 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm

real 0m12.916s
user 0m11.012s
sys 0m1.885s
```

[Alistair Buxton (a-j-buxton)]

It isn't clear to me why apt-check is so slow, but from systemctl I can see that it runs lsb-release to get the release name at least 10 to 20 times per invocation.

[Alistair Buxton (a-j-buxton)]

The result of this is usually that the user gets frustrated waiting for updates that never finish installing, so they just turn off the computer. Then next time there are even more updates, making he problem worse.