netplan generator segfaults, leaves server with no working DNS

[Marius Gedminas]

A server I have stopped being able to resolve DNS queries. According to resolvectl status, no DNS servers have been configured.

/etc/netplan/01-netcfg.yaml has networking configuration that looks correct to me, and if I do 'netplan try', I get working a network. Somehow systemd's configuration diverged from netplan's.

Before I did the 'netplan try', I looked at /run/systemd/network/. There was one file in it, /run/systemd/network/, with just

    [Match]

    [Link]
    WakeOnLan=magic

in it. The timestamp on that file is Apr 7, 06:28, which is almost 24 hours since the last reboot.

journalctl shows this at around that timestamp:

Apr 07 06:27:56 momsen systemd[1]: Starting Daily apt upgrade and clean activities...
Apr 07 06:27:56 momsen postfix/smtpd[38430]: disconnect from unknown[121.66.35.37] ehlo=1 auth=0/1 quit=1 commands=2/3
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen systemd[1]: Reloading.
Apr 07 06:28:01 momsen kernel: traps: netplan[38842] general protection fault ip:55c8cfed3cf0 sp:7fff25300030 error:0 in generate[55c8cfed0000+8000]
Apr 07 06:28:01 momsen systemd[38840]: /usr/lib/systemd/system-generators/netplan terminated by signal SEGV.
Apr 07 06:28:01 momsen systemd[1]: /lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/dbus/system_bus_socket → /run/dbus/system_bus_socket; please update the unit file accordingly.
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:01 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen dbus-daemon[548]: [system] Reloaded configuration
Apr 07 06:28:02 momsen systemd[1]: Reexecuting.
Apr 07 06:28:02 momsen systemd[1]: systemd 245.4-4ubuntu3.6 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
Apr 07 06:28:02 momsen systemd[1]: Detected architecture x86-64.
Apr 07 06:28:02 momsen kernel: traps: netplan[39030] general protection fault ip:563995b5acf0 sp:7ffc8299c2f0 error:0 in generate[563995b57000+8000]
Apr 07 06:28:02 momsen systemd[39028]: /usr/lib/systemd/system-generators/netplan terminated by signal SEGV.
...
Apr 07 06:28:03 momsen kernel: traps: netplan[39132] general protection fault ip:55f15ae8ecf0 sp:7fff9cbcb100 error:0 in generate[55f15ae8b000+8000]
Apr 07 06:28:03 momsen systemd[39130]: /usr/lib/systemd/system-generators/netplan terminated by signal SEGV.
...
Apr 07 06:28:07 momsen systemd-udevd[429]: /run/systemd/network/10-netplan-enp35s0.link: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add OriginalName=* in the [Match] section.
Apr 07 06:28:08 momsen systemd[1]: Reloading.
Apr 07 06:28:08 momsen kernel: traps: netplan[40203] general protection fault ip:55bbc1fe6cf0 sp:7ffedcbfea00 error:0 in generate[55bbc1fe3000+8000]
Apr 07 06:28:08 momsen systemd[40201]: /usr/lib/systemd/system-generators/netplan terminated by signal SEGV.
...
Apr 07 06:28:22 momsen systemd[1]: apt-daily-upgrade.service: Succeeded.
Apr 07 06:28:22 momsen systemd[1]: Finished Daily apt upgrade and clean activities.

/var/crash is empty, so I'm afraid I don't have any information about the netplan generator segfault. I tried running systemctl daemon-reload to see if I could provoke a crash again, and either that command doesn't run all the generators, or the netplan generator failed to crash. Or maybe something was broken during the nightly unattended upgrade on Apr 7 causing the crash -- e.g. I find it somewhat suspicious that /var/log/apt/history.log shows netplan.io and libnetplan0 being upgraded in separate transactions?

Start-Date: 2021-04-07 06:27:59
Commandline: /usr/bin/unattended-upgrade
Upgrade: libnetplan0:amd64 (0.101-0ubuntu3~20.04.2, 0.102-0ubuntu1~20.04.1)
End-Date: 2021-04-07 06:27:59

Start-Date: 2021-04-07 06:28:01
Commandline: /usr/bin/unattended-upgrade
Upgrade: libsystemd0:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), systemd-timesyncd:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), systemd-sysv:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), libpam-systemd:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), systemd:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), libnss-systemd:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6)
End-Date: 2021-04-07 06:28:05

Start-Date: 2021-04-07 06:28:07
Commandline: /usr/bin/unattended-upgrade
Upgrade: udev:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6), libudev1:amd64 (245.4-4ubuntu3.5, 245.4-4ubuntu3.6)
End-Date: 2021-04-07 06:28:18

Start-Date: 2021-04-07 06:28:20
Commandline: /usr/bin/unattended-upgrade
Upgrade: netplan.io:amd64 (0.101-0ubuntu3~20.04.2, 0.102-0ubuntu1~20.04.1)
End-Date: 2021-04-07 06:28:21

I'm sorry for the rambling bug report. I don't know how to make it better.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: netplan.io 0.102-0ubuntu1~20.04.1
ProcVersionSignature: Ubuntu 5.4.0-70.78-generic 5.4.94
Uname: Linux 5.4.0-70-generic x86_64
ApportVersion: 2.20.11-0ubuntu27.16
Architecture: amd64
CasperMD5CheckResult: skip
Date: Mon Apr 12 13:37:27 2021
ProcEnviron:
 LC_CTYPE=lt_LT.UTF-8
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: netplan.io
UpgradeStatus: No upgrade log present (probably fresh install)