systemd fails to set unit as inactive when using socket activation and the main process has exited

[Sergio Durigan Junior]

systemd 253.5 on Mantic is affected by a bug which makes it fail to mark a unit as inactive even when its main process exited (when using socket activation). This is affecting libvirt and possibly other services.

Upstream has a bug: https://github.com/systemd/systemd/issues/27953

which has been fixed by: https://github.com/systemd/systemd/pull/28000

To reproduce the problem:

$ lxc launch ubuntu-daily:mantic libvirt-hang --vm
$ lxc shell libvirt-hang
# apt update && apt upgrade -y
# apt install -y libvirt-daemon-system
# systemctl status libvirtd.service

You'll notice that there is a libvirt process running:

...
     CGroup: /system.slice/libvirtd.service
             ├─ 870 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
             ├─ 871 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
             └─1020 /usr/sbin/libvirtd --timeout 120
...

Wait for two minutes (or edit /etc/default/libvirtd and reduce the timeout), then check the status again. You'll notice that the libvirtd process has exited, but the unit is still marked as active:

root@libvirt-hang:~# systemctl status libvirtd.service
● libvirtd.service - Virtualization daemon
     Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; preset: enabled)
     Active: active (running) since Mon 2023-08-28 23:06:23 UTC; 57s ago
TriggeredBy: ● libvirtd-admin.socket
             ● libvirtd.socket
             ● libvirtd-ro.socket
       Docs: man:libvirtd(8)
             https://libvirt.org
    Process: 1020 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=0/SUCCESS)
   Main PID: 1020 (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 32768)
     Memory: 22.4M
        CPU: 161ms
     CGroup: /system.slice/libvirtd.service
             ├─870 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
             └─871 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
...

libvirtd.service is socket-activated, but the fact that it is still considered to be active after the main process exited means that the socket won't be actively listening, and you end up seeing libvirt-related commands hang indefinitely, effectively rendering libvirt useless until you manually restart the service.

[Nick Rosbrook (enr0n)]

This was indeed fixed in v253.6, so let's cherry pick the patch.

https://github.com/systemd/systemd-stable/commit/f29f0877c5abfd03060838d1812ea6fdff3b0b37

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 253.5-1ubuntu5

---------------
systemd (253.5-1ubuntu5) mantic; urgency=medium

  * Revert "core/service: when resetting PID also reset known flag" (LP: #2033325)
    Thanks to Sergio Durigan Junior
    File: debian/patches/lp2033325-Revert-core-service-when-resetting-PID-also-reset-known-f.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=021b6fafca7ae9bddaa479500614a4a4be40ce59
  * debian/rules: do not ship systemd-gpt-auto-generator.
    We do not want the functionality provided by this generator, so do not
    ship it.
    File: debian/rules
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a366279a84730e92ed734710560ea833709f359d

 -- Nick Rosbrook <email address hidden> Thu, 07 Sep 2023 16:09:54 -0400