systemd-resolved appends 127.0.0.53 to resolv.conf alongside existing entries

Bug #1624320 reported by Anders Kaseorg on 2016-09-16
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Low
Unassigned

Bug Description

systemd-resolved, or more precisely the hook script /lib/systemd/system/systemd-resolved.service.d/resolvconf.conf, causes resolvconf to add 127.0.0.53 to the set of nameservers in /etc/resolv.conf alongside the other nameservers. That makes no sense because systemd-resolved sets up 127.0.0.53 as a proxy for those other nameservers. The effect is similar to bug 1624071 but for applications doing their own DNS lookups. It breaks any DNSSEC validation that systemd-resolved tries to do; applications will failover to the other nameservers, bypassing validation failures. And it makes failing queries take twice as long.

/etc/resolv.conf should have only 127.0.0.53 when systemd-resolved is active.

Martin Pitt (pitti) wrote :

The primary purpose of adding 127.0.0.53 to resolv.conf is for client software that wants to do DNS resolution by itself instead of using NSS -- most notable example is Google Chrome, and third-party software which is statically linked (e. g. Go).

However, other software like NetworkManager or isc-dhcp also calls resolvconf and adds name servers picked up by them -- as they don't talk to resolved directly, resolved reads their DNS servers *from* resolv.conf.

But, software which does its own DNS lookups like the above have to do their own DNSSEC validation too -- you can't both chose to *not* use NSS *and* rely on NSS to do DNSSEC for you..

So, this is indeed a wart, but not easily fixed, and also not that important IMHO. Not using NSS is already broken to some degree, as you also ignore things like nss-{winbind,docker,ldap} etc.

Changed in systemd (Ubuntu):
status: New → Triaged
importance: Undecided → Low
Anders Kaseorg (andersk) wrote :

DNS resolution outside NSS shouldn’t be dismissed as an edge case for software that’s too conceited to use NSS. NSS only exposes A, AAAA, and PTR records. There’s plenty of software in the official archive that needs other records from DNS (off the top of my head: SRV, TXT, MX, SSHFP, AFSDB) and cannot get them from NSS.

> resolved reads their DNS servers *from* resolv.conf.

Right, so perhaps when resolvconf is in use, systemd-resolved should read those from /run/resolvconf/resolv.conf directly, and /etc/resolv.conf should be a symlink to /lib/systemd/resolv.conf rather than /run/resolvconf/resolv.conf?

> you can't both chose to *not* use NSS *and* rely on NSS to do DNSSEC for you.

Why not? It was working with NetworkManager managing a dnsmasq, since NetworkManager installed the local proxy as the only nameserver visible in resolv.conf, and it would work again if systemd-resolved did the same.

This will also be needed to fix problems like https://github.com/systemd/systemd/issues/3421 for programs that cannot use NSS.

Willem Toorop (willem-l) wrote :

Unfortunately the DNS interface of current systemd-resolved strips DNSSEC, so applications that do DANE validation still have to target the upstreams directly.
I have filed a bug about this: https://github.com/systemd/systemd/issues/4621

Martin Pitt (pitti) wrote :

As explained, 127.0.0.53 must be in /etc/resolv.conf in order to support Chromium and other software which does not NSS -- that is the whole raison d'être for providing a local stub server.

In Yakkety with NetworkManager only the 127.0.1.1 dnsmasq server is in /etc/resolv.conf, so that isn't affected. In Zesty NM now does not manager resolv.conf itself any more but just sends the acquired name servers to resolved.

Thus, is there anything left from the original bug report that still needs to be addressed?

Changed in systemd (Ubuntu):
status: Triaged → Incomplete
tags: added: resolved
Willem Toorop (willem-l) wrote :

Au contraire,

I argue to not fix this issue and keep the other upstream resolvers alongside 127.0.0.53 in /etc/resolv.conf, because 127.0.0.53 will break applications that need to do DNSSEC validation themselves (for example for DANE).

Once systemd-resolved has been fixed to provide the DNSSEC data alongside the answer, I agree with Anders that the other resolvers should go.

Anders Kaseorg (andersk) wrote :

Martin: I still wonder what the point of having resolvconf is, if it’s only ever supposed to be used to manage 127.0.0.53, and every other use of resolvconf will lead to this bug resurfacing. I still propose that systemd-resolved should read from /run/resolvconf/resolv.conf without adding 127.0.0.53 to it, and /etc/resolv.conf should be a symlink to /lib/systemd/resolv.conf rather than /run/resolvconf/resolv.conf.

Willem: If systemd-resolved isn’t suitable to be the only nameserver in resolv.conf, then it isn’t suitable to be in resolv.conf at all. I would rather either see these bugs worked out during the zesty cycle, or have systemd-resolved removed entirely, than leave the system in a half-broken state where the bugs in systemd-resolved get (probabilistically?) masked by the presence of other nameservers in resolv.conf.

In its current state, systemd-resolved is thoroughly broken on account of (at least) bug 1647031. I had to downgrade network-manager because switching to a resolver that doesn’t follow CNAME records breaks way too much of the internet. This might not have been noticed with other nameservers in resolv.conf. (Although now that it has, I’m getting increasingly concerned by the switch not having been reverted yet…)

Doug Goldstein (cardoe) wrote :

So this issue bit me in a weird way. I had my bridge called "xenbr0" and the result of these two interacting gives me absolutely no DNS resolution since the /etc/resolv.conf just pointed to 127.0.0.53 and didn't include the DNS server from my "xenbr0". Renaming the bridge to "renbr0" caused it to work. Why "renbr0"? Because in /run/resolvconf/interface/ there is a file called systemd-resolved that I was guessing was stopping combining the files once it hit the systemd-resolved.

Vincent Fortier (th0ma7) wrote :

From the man page systemd-resolve can run in 3 mode of operations.

1) Ubuntu 17.10 default - The default is to list the 127.0.0.53 DNS stub (see above) as only DNS server. This file may be symlinked from /etc/resolv.conf in order to connect all local clients that bypass local DNS APIs to systemd-resolved. This mode of operation is recommended.

2) WHAT YOU WANT: systemd-resolved maintains the /run/systemd/resolve/resolv.conf file for compatibility with traditional Linux programs. This file may be symlinked from /etc/resolv.conf and is always kept up-to-date, containing information about all known DNS servers....

3) PRE-17.04: Alternatively, /etc/resolv.conf may be managed by other packages, in which case systemd-resolved will read it for DNS configuration data. In this mode of operation systemd-resolved is consumer rather than provider of this configuration file.

The fix:
root@localhost:~# ls -la /etc/resolv.conf
lrwxrwxrwx 1 root root 29 mar 7 20:20 /etc/resolv.conf -> ../run/resolvconf/resolv.conf
root@localhost:~# rm -f /etc/resolv.conf
root@localhost:~# ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
root@localhost:~# ls -la /etc/resolv.conf
lrwxrwxrwx 1 root root 32 mar 8 07:30 /etc/resolv.conf -> /run/systemd/resolve/resolv.conf

Finally firefox started working properly along with all my command line tools...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.