replacement of resolvconf with systemd needs integration

[Scott Moser]

There is a plan to remove resolvconf from the Ubuntu Server image.
resolvconf integrated with other parts of the system in 2 ways:
 * hooks invoked on change (/etc/resolvconf/update.d/)
 * resolvconf tool (invoked with -a and -d or -u)

Packages which install files into /etc/resolvconf/update.d are:
- dnsmasq: This may be mostly covered by systemd-resolved itself (the dns
  caching path).
- resolvconf: This probably isn't necessary in systemd-resolved path.
- unbound: This is another "validating, recursive, caching DNS resolver".

The list of Depends/Suggests/Recommends on resolvconf.

# for pkg in $(apt-cache rdepends resolvconf | grep -v openreso | grep -v Reverse); do out=$(apt-cache show $pkg | grep resolvconf); src=$(apt-cache show $pkg | awk '$1 == "Source:" { print $2 }'); [ -n "$src" ] || src=$pkg; case "$out" in Depends:*resolvconf) r=depends;; Suggests:*) r=suggests;; Recommends:*) r=recommends;; esac; echo "$r $src"; done | sort -u
depends android-androresolvd
recommends avahi
recommends dhcpcd5
recommends dibbler
recommends ndisc6
recommends whereami
suggests bind9
suggests dnscrypt-proxy
suggests dnsmasq
suggests dnssec-trigger
suggests fetchmail
suggests freedombox-setup
suggests isc-dhcp
suggests netscript-2.4
suggests openvpn
suggests postfix
suggests pppconfig
suggests pump
suggests resolvconf
suggests sendmail
suggests squid3
suggests vpnc
suggests vpnc-scripts

ProblemType: Bug
DistroRelease: Ubuntu 17.10
Package: systemd 234-2ubuntu9
ProcVersionSignature: Ubuntu 4.12.0-11.12-generic 4.12.5
Uname: Linux 4.12.0-11-generic x86_64
ApportVersion: 2.20.6-0ubuntu7
Architecture: amd64
Date: Tue Aug 29 18:53:50 2017
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.12.0-11-generic root=UUID=f897b32a-eacf-4191-9717-844918947069 ro quiet splash vt.handoff=7
SourcePackage: systemd
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.vendor: Intel Corporation

Related bugs:
 * bug 1698181: Switch to netplan renderer in Artful
 * bug 1714308: dns does not work in initramfs after configure_networking
 * bug 1717983 replacement of isc-dhcp-client with with systemd-networkd for dhclient needs integration

[Scott Moser (smoser)]

I've added open-iscsi and cloud-initramfs-tools and initramfs-tools as also affects.
These packages are affected by the generic problem where 'configure_networking' from initramfs-tools is executed in the initramfs ('ip=dhcp' for example) and then the link is left up. In those cases we need to apply the /etc/resolv.conf changes from the initramfs to the "real root". I described how this *did* work for the open-iscsi package in the past, but there was no centralized handling of it (although there should have been).

[Scott Moser (smoser)]

At https://bugs.launchpad.net/ubuntu/+source/open-iscsi/+bug/1713537/comments/1 I described how open-iscsi (iscsi root) works with resolvconf in xenial -> zesty. We will need a solution for that path as well as 'root=squashfs:http://...../squashfs' that MAAS now uses.

It seems like we should have a general path that covers
 a.) dns working in initramfs (currently dns does not, it is missing libnss_dns.so.2)
 b.) network interfaces that are left up from initramfs getting resolvconf populated in the "real root".

My idea for this was just to have initramfs's configure_networking write a file in /run/network/ that was then read by systemd-resolved when it started.

[Scott Moser (smoser)]

I filed bug 1714308 to cover dns functionality in the initramfs.

[Steve Langasek (vorlon)]

avahi's hook is for purposes of disabling .local resolution in the event that a .local domain becomes available as a DNS search domain. This is the unresolved integration/design issue LP: #327362.

[Steve Langasek (vorlon)]

android-androresolvd: "This only makes sense to install on a Debian chroot running on Android."

[Steve Langasek (vorlon)]

bind9 integrates with resolvconf to register itself as a DNS server. It is reasonable to expect something similar for integration with resolved.

[Steve Langasek (vorlon)]

dibbler-client doesn't interface with resolvconf, it only recommends it. This does not warrant carrying an Ubuntu delta.

[Steve Langasek (vorlon)]

isc-dhcp-client only suggests resolvconf, it does not interface with it.

[Steve Langasek (vorlon)]

netscript-2.4 is an alternate network manager package which would make its own technology selections independent of Ubuntu defaults and systemd.

[Dimitri John Ledkov (xnox)]

isc-dhcp now integrates with resolved.

[Dimitri John Ledkov (xnox)]

i see no direct calls to resolvconf or hooks installed by cloud-init. Marking incomplete.

[Brian Murray (brian-murray)]

fetchmail suggests resolvconf and does not actually utilize it.

[Steve Langasek (vorlon)]

fetchmail integrates with resolvconf via /etc/resolvconf/update-libc.d/fetchmail; I think this warrants another look.

[Brian Murray (brian-murray)]

I've looked more closely at fetchmail and while the resolvconf integration isn't critical to it working it does improve fetchmail, so I'm reopening the bug task althought it is a minor issue.

[Brian Murray (brian-murray)]

dnscrypt-proxy is another package which suggests resolvconf and one where the integration is not critical. It also seems like a minor issue.

[Brian Murray (brian-murray)]

dnssec-trigger actually has and has had a Breaks with resolvconf so that task seems unnecessary.

[Brian Murray (brian-murray)]

pump also suggests resolvconf but the pump binary is modified to have a --no-resolvconf switch so this seems easy to workaround. It's worth noting that the resolvconf support is a separate patch in debian/patches so would be easy to drop.

[Brian Murray (brian-murray)]

Integrating openvpn with resolvconf requires the user to take manual action and modify their openvpn .conf file to call /etc/openvpn/update-resolv-conf. That being said it'd be good to help users who rely on this and are upgrading.

[Brian Murray (brian-murray)]

pppconfig seems to fall in the same category as dnssec-trigger and dnscrypt-proxy.

[Steve Langasek (vorlon)]

On Fri, Feb 09, 2018 at 10:29:57PM -0000, Brian Murray wrote:
> pump also suggests resolvconf but the pump binary is modified to have a
> --no-resolvconf switch so this seems easy to workaround. It's worth
> noting that the resolvconf support is a separate patch in debian/patches
> so would be easy to drop.

pump is also a dhcp/bootp client, which is entirely redundant with the *two*
dhcp client implementations in main and installed by default.

[Brian Murray (brian-murray)]

Ubuntu has long carried a patch which "Eliminate all references to /etc/resolvconf/run." However, freedombox-setup contains the following code in a preseed file.

# Make sure DNS lookup work after resolvconf is installed
# and set up the Freedombox. Block init.d scripts from running using
# policy-rc.d to make sure dnsmasq do not overwrite the resolv.conf
# file we just inserted (and to keep services from starting in the chroot).
d-i preseed/late_command string cp /etc/resolv.conf /target/etc/resolvconf/run/resolv.conf

That's not likely to work and has probably been broken for a long time (without a bug report). So while freedombox-setup needs fixing it doesn't seem important to anybody.

[Mathieu Trudel-Lapierre (cyphermox)]

ndisc6 appears to Recommends: resolvconf (in rdnssd); will drop to a Suggests. The code in the merge hook for resolvconf already checks whether resolvconf is present before making use of it.

[Mathieu Trudel-Lapierre (cyphermox)]

It looks like open-iscsi makes use of resolvconf in debian/net-interface-handler, but does check for its presence before doing so. That said, there is not additional integration with anything to set nameservers/search domains, so open-iscsi will need further investigation to see if all is well with it when resolvconf is not present and the iSCSI server is configured with an FQDN rather than a straight IP address.

[Mathieu Trudel-Lapierre (cyphermox)]

vpnc and vpnc-scripts check for existance of /sbin/resolvconf and only Suggests: resolvconf. DNS integration by modifying /etc/resolv.conf (and thus the systemd symlink) should work appropriately and let systemd-resolved know about the new nameservers.

[Mathieu Trudel-Lapierre (cyphermox)]

dnsmasq Suggests: resolvconf only, and checks before using it. None of the steps involved would break with resolvconf not being present, integration just happens via /etc/resolv.conf normally.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ndisc6 - 1.0.3-3ubuntu1

---------------
ndisc6 (1.0.3-3ubuntu1) bionic; urgency=medium

  * debian/control: Drop resolvconf to a Suggests:. It's not absolutely
    necessary to have resolvconf installed to use rdnssd, and resolvconf
    might fight with systemd-resolved for ownership of /etc/resolv.conf.
    (LP: #1713803)

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 16 Feb 2018 13:18:54 -0500

[Steve Langasek (vorlon)]

Marking open-iscsi high, because passing the configured network interface from initramfs to root system needs to be done cleanly in the non-resolvconf world (though this is likely going to be handled via netplan rather than directly to resolved).

[Michael Hudson-Doyle (mwhudson)]

I had a quick look at unbound, it seems quite difficult. People who install unbound probably want it so they can have DNSSEC supported in their local resolver which we can't really do by integrating with resolved. OTOH, if installing unbound replaces (in some sense) resolved and everything else integrates with resolved, that has problems too.

[Mathieu Trudel-Lapierre (cyphermox)]

dhcpcd5 is redundant as a DHCP implementation in Ubuntu; but it does "integrate" with resolvconf by way of its own shipped dhcpcd-hooks/20-resolv.conf. It does check before calling 'resolvconf -a' and 'resolvconf -d' to add/remove its own interface file; and uses this hook file in order to integrate nicely with VPN clients and/or local nameservers.

Setting to Confirmed, the package should be fixed eventually to play nice with systemd-resolved.

[Mathieu Trudel-Lapierre (cyphermox)]

postfix takes a copy of resolv.conf for its own use, in its own queue directory. This is already well integrated by way of either resolvconf triggering the copy on changes (using update-libc.d) or by if-up / if-down. This will need work to "improve" the integration story; however postfix should still work well in its current state with systemd given that the effective nameserver should remain just the local systemd-resolved.

[Mathieu Trudel-Lapierre (cyphermox)]

Looks like the current behavior should be sufficient for postfix to integrate as well as it did with resolvconf: marking this Invalid.

[Mathieu Trudel-Lapierre (cyphermox)]

sendmail only uses an update-libc.d script to reload the daemon on changes to resolv.conf, which should be sufficient for DNS resolution to remain working as it did with resolvconf; seeing as the nameserver will generally not change from 127.0.0.53

[Mathieu Trudel-Lapierre (cyphermox)]

squid3: same story as sendmail, uses update-libc.d to get notified of nameserver changes.

[Mathieu Trudel-Lapierre (cyphermox)]

unbound attempts to add itself as a local resolver (pointing to 127.0.0.1). I think this will require specific integration work so that unbound can properly update/ tell systemd-networkd that it wants to be able to serve as a resolver.

[Launchpad Janitor (janitor)]

This bug was fixed in the package initramfs-tools - 0.130ubuntu6

---------------
initramfs-tools (0.130ubuntu6) cosmic; urgency=medium

  * Fix formatting for the generated netplan config: headers were missing.

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 10 May 2018 12:12:03 -0400

[Łukasz Zemczak (sil2100)]

How will the initramfs-tools change be validated? What test case will be run to make sure it's working as intended?

[Martin-Éric Racine (q-funk)]

Does dhcpcd5 version 9.4.1-0.1 solve the issue for you?

[Steve Langasek (vorlon)]

The resolvconf interface is now implemented in the systemd-resolved package and integrates correctly with systemd. There is no longer a need for packages to port away from this interface.