systemd-resolved.service hangs a long time on shutdown

[Martin Pitt]

On shutdown or "systemctl stop systemd-resolved" you get a long hang:

● systemd-resolved.service - Network Name Resolution
   CGroup: /system.slice/systemd-resolved.service
           └─control
             ├─15479 /bin/sh -c [ ! -e /run/resolvconf/enable-updates ] || /sbin/resolvconf -d systemd-resolved
             ├─15480 run-parts --arg=-d --arg=systemd-resolved /etc/resolvconf/update.d
             ├─15483 run-parts /etc/resolvconf/update-libc.d
             ├─15497 /bin/sh /usr/lib/avahi/avahi-daemon-check-dns.sh
             └─15509 host -t soa local.

So that resolvconf hook tries to do name resolution which does not work any more at that time.

[Martin Pitt (pitti)]

Unfortunately resolvconf does not have a --no-scripts or similar option that would disable running the update.d/ hooks. One possible local workaround is to change /lib/systemd/system/systemd-resolved.service.d/resolvconf.conf from

  ExecStopPost=+/bin/sh -c '[ ! -e /run/resolvconf/enable-updates ] || /sbin/resolvconf -d systemd-resolved'

to

  ExecStopPost+=/bin/rm -f /run/resolvconf/interface/systemd-resolved

or to drop the line completely.

This solves the hang on shutdown, but it does not drop 127.0.0.53 any more from /etc/resolv.conf if you manually stop systemd-resolved.service in a running system.

This should actually happen the same way with dnsmasq or any other local DNS server -- if only that is in resolv.conf, then the Avahi hook script would run into this timeout on "host" as well, as the local name server is already gone. Our workaround for that in 16.04 was to never stop dnsmasq even when NetworkManager.service got stopped (via KillMode=process). However, when you do stop dnsmasq then you get similar hangs with trying to do DNS queries.

At the moment, if you stop the local DNS server then there is nothing that would magically bring back the non-local DNS servers into resolv.conf (neither in zesty with resolved nor in 16.04 with dnsmasq), so you would run into timeouts either way. Thus I think just dropping the ExecStopPost= does not actually make things worse, but it fixes the hang on shutdown.

[Martin Pitt (pitti)]

https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/commit/?id=dbda116b2

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 232-8

---------------
systemd (232-8) unstable; urgency=medium

  [ Martin Pitt ]
  * Drop systemd dependency from libnss-myhostname again.
    This NSS module is completely independent from systemd, unlike the other
    three.
  * Install 71-seat.rules into the initrd.
    This helps plymouth to detect applicable devices. (Closes: #756109)
  * networkd: Fix crash when setting routes.
  * resolved: Drop removal of resolvconf entry on stop.
    This leads to timeouts on shutdown via the resolvconf hooks and does not
    actually help much -- /etc/resolv.conf would then just be empty instead of
    having a nonexisting 127.0.0.53 nameserver, so manually stopping resolved
    in a running system is broken either way. (LP: #1648068)
  * Keep RestrictAddressFamilies on amd64.
    This option and libseccomp currently work on amd64 at least, so let's make
    sure it does not break there as well, and benefit from the additional
    protection at least on this architecture.
  * Explicitly set D-Bus policy dir.
    This is about to change upstream in
    https://github.com/systemd/systemd/pull/4892, but as explained in commit
    2edb1e16fb12f4 we need to keep the policies in /etc/ until stretch+1.

  [ Michael Biebl ]
  * doc: Clarify NoNewPrivileges in systemd.exec(5). (Closes: #756604)
  * core: Rework logic to determine when we decide to add automatic deps for
    mounts. This adds a concept of "extrinsic" mounts. If mounts are
    extrinsic we consider them managed by something else and do not add
    automatic ordering against umount.target, local-fs.target,
    remote-fs.target. (Closes: #818978)
  * rules: Add persistent links for nbd devices. (Closes: #837999)

 -- Michael Biebl <email address hidden> Sat, 17 Dec 2016 01:54:18 +0100