nova fails to configure dnsmasq, resulting in DNS timeouts in instances

[LaMont Jones]

Nova uses dnsmasq to answer questions about name <-> IP resolution for instances. By default, it does nothing about things where there is no answer.

This causes dnsmasq to forward the query (for which it should be authoritative) off to the nameserver found in resolv.conf. If the zone is properly delegated to nova via a forward only zone declaration in the resolver, then we run into the situation where the instance asks dnsmasq which asks the resolver which asks dnsmasq which then times out.

Combine this with linux' love for IPv6, and a single domain search list in resolv.conf, and anything that looks up a host name (e.g., sudo) will take 10 seconds (5 seconds each for the lookup of $(hostname).$domain and $(hostname) AAAA RRs), before it fails back to looking up $(hostname).$domain A RR and gets an answer.

The fix that worked for us was to add --dnsmasq_config_file=/etc/nova/dnsmasq.conf (not --dns_server, because we DO NOT WANT -h and -R passed to dnsmasq, and we need to specify multiple --server directives) and then dnsmasq.conf gets
"--server=/xxx.yyy.10.in-addr.arpa/ --server=/openstack.example.com/" which tells it to not forward queries for those zones off-machine. (The lack of -h and -R means that we do not break our ability to resolute the rest of the DNS world.)

[Chuck Short (zulcss)]

So there needs an option to have multiple dns servers correct?

[LaMont Jones (lamont)]

As discussed in IRC:

If the domain I assign to nova is its own (which I believe it should be), then it should be authoritative for that zone. As in it shouldn't go asking the upstream resolver to give it answers for things that are delegated to it. This is also true of the reverse zone for the IP blocks that are assigned to nova networks (public and private).

That is, forward and reverse zones should be handed to dnsmasq in server= declarations based on the network and DNS config of the network. Alternatively, running an actual nameserver would be an option, as would having nova-network use nsupdate to update such a beast.

--dns_server doesn't do the trick, since (a) multiple domains are involved and (b) nova already has the answers - making the admin tell it twice is just asking for errors.

Please let me know if there are any more questions on the subject.

[Joe Gordon (jogo)]

Is this still valid? this bug is very old

[Sean Dague (sdague)]

In looking at the code I feel like this is effectively addressed in the current linux_net.py, please reopen if it's still an issue.

[Bryan Quigley (bryanquigley)]

Closed upstream as being fixed for a while - closing Ubuntu task as well.