apt hook slows down apt - could this be asynchronous?

[Christian Ehrhardt ]

Hi,
I was actually looking for needrestart as it has the last messages in an apt transaction and then seems to stall. But I found that it isn't
  /etc/apt/apt.conf.d/99needrestart
instead it is the other 99 which is
  /etc/apt/apt.conf.d/99update-notifier

I've littered the steps it executes for debugging and found that the final call to `/usr/lib/update-notifier/apt-check --human-readable` takes about 4 seconds.

Here at the end of an `apt install --reinstall` of any package.

DEBUG I 07:04:54.330869746
DEBUG II 07:04:54.331734979
DEBUG III 07:04:54.350096999
DEBUG IV 07:04:54.360757929
DEBUG V 07:04:54.376343616
DEBUG VI 07:04:54.379049099
DEBUG VII 07:04:54.386505261
DEBUG VIII 07:04:58.824188091 <<< those 4 seconds
DEBUG IX 07:04:58.826370395
DEBUG IX 07:04:58.827825319

The time varies on system HW and installed packages, but is always at an annoying >1sec.
If not just an automated background install people often wait for apt to complete to be able to do the next steps.
And then apt seeming complete (it is actually) but having this artificial delay makes Ubuntu feel slow every time.

This is only meant to populate /var/lib/update-notifier/updates-available with some content to be used by the notifier later. I don't know if there are other consumers of this file, but you'd think that the motd infrastructure would certainly be ok if this is complete slightly after the apt transaction.

Somewhere in the back of my mind was that this was tried ...
And what makes me suspicious is at least:
 # let the actual update be asynchronous to avoid stalling apt-get
 cleanup() { rm -f "$tmpfile"; }

I checked the change-log for it and found bug 1527710 which explains why - currently - it isn't asynchronous anymore. (The changelog also explains why I thought to remember this being async, I did that myself 6 years ago). I have to agree all those issues with the asynchronicity back then were bad, but while the current stall isn't as "breaking" is is really bad on the "user inconvenience" side of things.

I wanted to file this bug (despite it being tried in the past) to discuss if we could find a way to disconnect these and make apt fast again.

[Christian Ehrhardt  (paelzer)]

How about fully decoupling the hook and the check?

Like letting /usr/lib/update-notifier/update-motd-updates-available only do the post-apt processing if a check "is needed" and then touch a stamp or similar.
That would give apt the fast path we want to have.

Independently we could then have something that regularly checks if the stamp exists or changed - and if so run the actual "/usr/lib/update-notifier/apt-check --human-readable" to output to "/var/lib/update-notifier/updates-available" which is what motd will later consume.

I think nowadays we can do better than "check every x minutes" like thtough systemd https://www.freedesktop.org/software/systemd/man/systemd.path.html.

Imagine (untested suggestion) instead of writing to:
  /usr/lib/update-notifier/update-motd-updates-available
it would only write the output of `date` to:
  /usr/lib/update-notifier/update-motd-updates-available-needcheck

Then we could have:
$ cat /etc/systemd/system/update-motd-updates-available.service
[Unit]
Description=Update list of available updates for update-notifier
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/lib/update-notifier/apt-check --human-readable
StandardOutput=file:/usr/lib/update-notifier/update-motd-updates-available

$ cat /etc/systemd/system/update-motd-updates-available.path
[Path]
PathModified=/usr/lib/update-notifier/update-motd-updates-available-needcheck

[Install]
WantedBy=multi-user.target

I'd hope that this would achieve
a) apt transactions being fast
b) the info of available packages will be updated immediately
c) the old issues of 1527710 were around mount/chroot/umount and similar.
   In those the service would not run -> no conflict
d) since there is no direct child process other old issues e.g. preseeding
   should also not be affected anymore

Opinions?

[Christian Ehrhardt  (paelzer)]

i#m not entirely sure on PathChanged= vs PathModified= but want to hear an opinion on the general approach first.

[Haw Loeung (hloeung)]

Something similar was done for checking for the latest Ubuntu release and updating the MOTD in LP:1836475. It switches the download part into a systemd timer (https://launchpadlibrarian.net/505544658/lp1836475-focal.debdiff). So maybe something similar here where the timer job checks existence and timestamp of a file and runs if it doesn't exist or is older than a certain period of time.

Then the apt hook itself would remove that file forcing the systemd timer job to fire.