5s delay in AAAA dns resolving; artful and earlier is quick

[Andreas Hasenack]

Basic steps:
lxc launch ubuntu-daily:bionic bionic-daily
lxc exec bionic-daily bash
sleep 10s (because of #1765173)
time host -d bionic-daily.lxd # or whatever domain your containers use by default

In bionic, there is a 5s delay when requesting the AAAA record:

root@wondrous-grackle:~# time host -d wondrous-grackle.lxd
Trying "wondrous-grackle.lxd"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50934
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;wondrous-grackle.lxd. IN A

;; ANSWER SECTION:
wondrous-grackle.lxd. 0 IN A 10.0.100.125

Received 54 bytes from 127.0.0.53#53 in 0 ms
Trying "wondrous-grackle.lxd"

(5s delay here)

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17153
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;wondrous-grackle.lxd. IN AAAA

Received 38 bytes from 127.0.0.53#53 in 1 ms
Trying "wondrous-grackle.lxd"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12145
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;wondrous-grackle.lxd. IN MX

Received 38 bytes from 127.0.0.53#53 in 0 ms

real 0m5.023s

In artful and xenial, there is no such delay and the (correct) empty reply is almost instantaneous.

See https://pastebin.ubuntu.com/p/KP5DBcKyJr/

In both cases, /etc/resolv.conf points at 127.0.0.53

systemd version:
 *** 237-3ubuntu8 500
        500 http://br.archive.ubuntu.com/ubuntu bionic/main amd64 Packages

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[carl2187 (carl2187)]

I'm having similar problems with DNS resolution in Bionic Ubuntu-Server.

After configuring the netplan file, with the "local" search suffix, name resolution works, but I see a 5 second delay for any DNS name ending in .local.

When watching wireshark during the DIG request, the local system makes about 25 successful DNS queries to the correct dns server, and the correct response is returned for all 25 queries. This happens in seconds 0 to 1. Then there is a pause of 4 seconds, then an identical query and response occurs at second 5, then the DIG result is returned, and the response is cached by systemd-resolvd.

For some reason the DNS resolver is ignoring all 25 of its first attempts to resolve the name in the first 0-1 seconds. Then it waits for 4 seconds, makes a single query, and this result is accepted and used.

This netplan file is the only configuration change after installation of Bionic 18.04.2 Server:

$ cat /etc/netplan/01-netcfg.yaml
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
 ethernets:
  eth0:
   addresses: [192.168.9.2/24]
   gateway4: 192.168.9.254
   nameservers:
    search: [local]
    addresses: [192.168.9.1]
 version: 2

[carl2187 (carl2187)]

Any five second delay in name resolution is most likely caused by this open bug in systemd-resolved:

https://github.com/systemd/systemd/issues/11849

The root cause is a delay when resolved has to downgrade the feature set of its requests to upstream DNS servers. It correctly discovers a compatible feature set, but doesn't implement the discovered feature set until the next new query attempt. The retry logic is supposed to handle this, but the bug is that it goes crazy and retries with a known not working feature set until the max retry counter is hit. Then 5 seconds later, a new attempt is made, and works perfectly, the compatible feature set is used on the first attempt of the new query.

[Alberto Donato (ack)]

I'm seeing this in eoan, too.

Whenever I try to access a host in my lan, host resolution takes 15s (tried timing "ping -c 1" and "nslookup"

a "dig <host>" query is immediate, but a "dig AAAA <host>" also takes 15s.

It seems it's trying 3 times, each with a 5s timeout before failing.

[Dan Streetman (ddstreet)]

please reopen if this is still an issue