dnsmasq crash when no servers in resolv.conf
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
dnsmasq (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Undecided
|
Andreas Hasenack |
Bug Description
[ Impact ]
dnsmasq "keeps an eye" on /etc/resolv.conf, and reloads it whenever the file is updated. When that happens and for some reason there were no "nameserver" declarations in the updated file, dnsmasq can crash.
Here is a log of a reproducer:
$ dig +short @127.0.0.1 ubuntu.com
;; communications error to 127.0.0.1#53: timed out
;; communications error to 127.0.0.1#53: connection refused
;; communications error to 127.0.0.1#53: connection refused
;; no servers could be reached
We can see the startup, then when resolv.conf is read again and no nameservers were found, and the crash:
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: started, version 2.86 cachesize 150
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: DNS service limited to local subnets
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: reading /etc/resolv.conf
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: using nameserver 10.0.100.1#53
Jan 03 13:57:13 j-dnsmasq-2045570 dnsmasq[1507]: read /etc/hosts - 7 addresses
Jan 03 13:57:13 j-dnsmasq-2045570 systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.
Jan 03 13:58:01 j-dnsmasq-2045570 dnsmasq[1507]: no servers found in /etc/resolv.conf, will retry
Jan 03 13:58:22 j-dnsmasq-2045570 systemd[1]: dnsmasq.service: Main process exited, code=dumped, status=11/SEGV
Jan 03 13:58:22 j-dnsmasq-2045570 systemd[1]: dnsmasq.service: Failed with result 'core-dump'.
dnsmasq has provisions for this situation, we can see that in the 13:58:01 message where it says it will retry, but due to this bug, it crashes instead.
The problem was introduced[1] in version 2.86, and fixed in 2.87, so only jammy is affected.
1. https:/
The commit message says:
"""
This problem was introduced in 2.86.
"""
And indeed, I wasn't able to crash 2.80 shipped in focal.
[ Test Plan ]
It might take a few tries to reproduce the bug, but here is the general outline. Also keep in mind that it's important to use a DNS name that isn't cached already by a previous query.
# Create a jammy lxd container
lxc launch ubuntu-daily:jammy j-dnsmasq-2045570
# Enter the container
lxc shell j-dnsmasq-2045570
# From now on, all commands should be executed in the container.
# Install dnsmasq, and disable systemd-resolved
apt update && apt install -y dnsmasq
# Disable systemd-resolved, and start dnsmasq
systemctl disable --now systemd-resolved
systemctl enable --now dnsmasq
# In one terminal inside the container, watch the dnsmasq logs:
journalctl -u dnsmasq.service -f
# In another terminal, remove /etc/resolv.conf and create a new one, empty
rm /etc/resolv.conf
echo "nameserver 1.1.1.1" > /etc/resolv.conf
# restart dnsmasq
systemctl restart dnsmasq.service
# Perform a dns query
dig @127.0.0.1 +short linux.com
# Comment the namserver directive in resolv.conf
echo "#nameserver 1.1.1.1" > /etc/resolv.conf
# Observe in the dnsmasq logs that it notices the change with a message like:
Jan 03 14:14:51 j-dnsmasq-2045570 dnsmasq[2274]: no servers found in /etc/resolv.conf, will retry
# Perform a *different* DNS query
dig @127.0.0.1 +short ubuntu.com
# Observe in the dnsmasq logs that it crashes.
Jan 03 13:58:22 j-dnsmasq-2045570 systemd[1]: dnsmasq.service: Main process exited, code=dumped, status=11/SEGV
Jan 03 13:58:22 j-dnsmasq-2045570 systemd[1]: dnsmasq.service: Failed with result 'core-dump'.
If it doesn't crash right away, repeat these steps a few times, but using a different domain name each time:
- add "nameserver 127.0.0.1" to /etc/resolv.conf
- observe that dnsmasq notices the change to the file
- perform a query for some random domain using "dig @127.0.0.1 +short <domain-
- remove "nameserver" from /etc/resolv.conf, observe that dnsmasq noticed the change
- perform a query for another random domain
The fixed version from proposed will not crash. That last query with no "nameserver" lines in resolv.conf won't work, but it won't crash the server.
[ Where problems could occur ]
This is doing some pointer/memory manipulation that could introduce memory leaks or other crashes. In fact, this is exactly what happened in the 2.86 release, which, and I quote, "Major rewrite of the DNS server and domain handling code. This should be largely transparent, but it drastically improves performance and reduces memory foot-print"[2]. 2.88 was then released with the fix used in this SRU (the commit is also in the 2.87 tag, but the upstream release notes only mention it in 2.88).
[ Other Info ]
Not at this time.
[ Original description ]
upstream discussion: https:/
in my journal, my dns service crash and restart just after:
Dec 04 17:18:38 dnsmasq[199333]: no servers found in /run/NetworkMan
oops report: https:/
ubuntu jammy, dnsmasq-base 2.86-1.1ubuntu0.3
Related branches
- git-ubuntu bot: Approve
- Athos Ribeiro (community): Approve
- Canonical Server Reporter: Pending requested
-
Diff: 31 lines (+12/-0)2 files modifieddebian/changelog (+7/-0)
src/dnsmasq.c (+5/-0)
description: | updated |
Changed in dnsmasq (Ubuntu Jammy): | |
assignee: | nobody → Andreas Hasenack (ahasenack) |
description: | updated |
Changed in dnsmasq (Ubuntu Jammy): | |
status: | Triaged → In Progress |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
description: | updated |
tags: |
added: verification-done removed: verification-needed |
Thanks for taking the time to report this bug and trying to make Ubuntu better.
Also thanks for the pointers. According to the upstream discussion this is the needed fix:
https:/ /thekelleys. org.uk/ gitweb/ ?p=dnsmasq. git;a=commit; h=d290630d31f45 17ab26392d00753 d1397f9a4114
It is included in version 2.87 onward, so it affects only Jammy.