Ordering of mdns4_minimal and resolve in /etc/nsswitch.conf causes mDNS lookups to fail -- breaks network printing

[Mark Tearle]

(See also libnss-resolve:amd64 231-9ubuntu1 amd64 nss module to resolve names via systemd-resolved)

# fresh install of yakkety

mtearle@liberation:/etc$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.10
Release: 16.10
Codename: yakkety

# package details

mtearle@liberation:~$ apt-cache policy libnss-resolve
libnss-resolve:
  Installed: 231-9ubuntu1
  Candidate: 231-9ubuntu1
  Version table:
 *** 231-9ubuntu1 500
        500 http://mirror.rackspace.com/ubuntu yakkety-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     231-9git1 500
        500 http://mirror.rackspace.com/ubuntu yakkety/main amd64 Packages
mtearle@liberation:~$ apt-cache policy systemd
systemd:
  Installed: 231-9ubuntu1
  Candidate: 231-9ubuntu1
  Version table:
 *** 231-9ubuntu1 500
        500 http://mirror.rackspace.com/ubuntu yakkety-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     231-9git1 500
        500 http://mirror.rackspace.com/ubuntu yakkety/main amd64 Packages

# attempt to ping VM elsewhere on network with mDNS hostname

mtearle@liberation:/etc$ ping bazzavan.local
ping: bazzavan.local: Name or service not known

# can find both ipv4 and ipv6 addresses for the host

mtearle@liberation:/etc$ avahi-resolve-host-name bazzavan.local
bazzavan.local fe80::a00:27ff:fea5:3f51

mtearle@liberation:/etc$ avahi-resolve-host-name -4 bazzavan.local
bazzavan.local 172.16.44.48

# can ping it

mtearle@liberation:/etc$ ping -c 1 172.16.44.48
PING 172.16.44.48 (172.16.44.48) 56(84) bytes of data.
64 bytes from 172.16.44.48: icmp_seq=1 ttl=64 time=0.265 ms

--- 172.16.44.48 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.265/0.265/0.265/0.000 ms

# original ordering

mtearle@liberation:/etc$ grep hosts /etc/nsswitch.conf
hosts: files resolve [!UNAVAIL=return] mdns4_minimal [NOTFOUND=return] dns

# go away and edit /etc/nsswitch.conf
# change ordering of resolve and mdns4_minimal

mtearle@liberation:/etc$ grep hosts /etc/nsswitch.conf
hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns

# check mdns lookups now work, and it now pings

mtearle@liberation:/etc$ ping -c 1 bazzavan.local
PING bazzavan.local (172.16.44.48) 56(84) bytes of data.
64 bytes from 172.16.44.48 (172.16.44.48): icmp_seq=1 ttl=64 time=0.161 ms

--- bazzavan.local ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.161/0.161/0.161/0.000 ms

# check libnss-resolve is still doing its thing

mtearle@liberation:/etc$ ping -c 1 localhost.localdomain
PING localhost.localdomain(localhost (::1%1)) 56 data bytes
64 bytes from localhost (::1): icmp_seq=1 ttl=64 time=0.016 ms

--- localhost.localdomain ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.016/0.016/0.016/0.000 ms

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[nitrogen (i-am-nitrogen)]

Some other reports of this bug from around the web, for information+confirmation:

https://ubuntuforums.org/showthread.php?t=2340042&p=13557627#post13557627

https://ubuntuforums.org/showthread.php?t=2340042&p=13557382#post13557382

https://askubuntu.com/questions/842771/avahi-dns-doesnt-work

https://askubuntu.com/questions/837982/how-to-configure-local-dns-lookup-in-ubuntu-16-10

https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1624071/comments/7

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Anton Keks (anton-azib)]

Quite easy to fix.
In Ubuntu 16.10 mdns is broken for everyone.
Please release the fix!

[Alberto Donato (ack)]

Any update on this bug?

[Mark Tearle (mark-tearle)]

Just did a quick test with a clean install of zesty + libnss-mdns

Installs with a correct ordering of resolvers in /etc/nsswitch.conf and works correctly.

[Robie Basak (racb)]

Hi Mark!

Thank you for pointing me to this bug. I spent some time looking into this.

It seems that there is currently no established mechanism for ordering NSS modules into nsswitch.conf automatically. Instead every package is doing its own parsing mangling of the file on install. libnss-resolve is appending "[!UNAVAIL=return]" which causes the mdns4_minimal entry to get shadowed, depending on installation order of the packages involved (libnss-mdns and libnss-resolve).

A workaround was implemented in Debian in https://anonscm.debian.org/cgit/collab-maint/nss-mdns.git/commit/?id=44550bcc6b49116f927dea52395867ff47d7d3b9 (tests in https://anonscm.debian.org/cgit/collab-maint/nss-mdns.git/commit/?id=0e9b82b91cd9bf9464cf63df2c3f1cbbec3553e0) which appears in Zesty (17.04) which will explain why Zesty appears to work correctly. Though this is still a hack, I think it should work in both ordering cases, so in the short term this should be fixed from a user's perspective.

In the long term, Debian have a bug open (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=846944) to fix the hack. It may be that mDNS resolution moves to libnss-resolve, or that some kind of configuration wrapper is arranged to improve on the ad-hoc mangling of nsswitch.conf.

So, for Ubuntu:

This is fixed in Zesty onwards.

The workaround for Yakkety (as you explained) is to edit /etc/nsswitch.conf and bring "mdns4_minimal [NOTFOUND=return]" in front of "resolve [!UNAVAIL=return]".

We could propose an update for Yakkety (16.10). But we have to do it carefully - if there is a bug in the update, we could break DNS resolution for all users. However, Debian have already written the essential "swap it round" code so that will already have had some real world testing. But Yakkety has only two months left before it EOLs, and the workaround for affected users is fairly trivial. So is it worth trying? Opinions welcome.

For bug triage, there are effectively two bugs. 1) The fact that it doesn't work for users in Yakkety; and 2) the bigger picture fix for the hack or move of mDNS resolution to libnss-resolve. I see this bug as having been reported for the first issue, and the Debian bug as tracking the second. So I'll mark the systemd task as Invalid, as the fix for this bug was in libnss-mdns (and we can track any SRU for libnss-mdns here). I'm not linking the Debian bug, as that's the separate second issue. If we want to track that in Launchpad, we should file a separate bug for it, so as not to mix up the separate statuses of the two issues.

[Mark Tearle (mark-tearle)]

Hi Robie

I did a quick test of a upgrade from Yakkety to Zesty. The post install script fixes things as expected. (Broken on Yakkety to Working on Zesty)

From a users perspective then;
1/ Advanced users will likely find the bug report and undertake the fix themselves
2/ A solid workaround of doing a release-upgrade to Zesty exists (which should fix things)

This only leaves users remaining behind on Yakkety who lack the technical skill to fix the problem as affected.