Activity log for bug #1981794

Date Who What changed Old value New value Message
2022-07-15 06:28:20 Reuben Lifshay bug added bug
2022-09-29 20:39:59 Lena Voytek attachment added fix-dns-retry-confusion-jammy.patch https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1981794/+attachment/5620120/+files/fix-dns-retry-confusion-jammy.patch
2022-09-29 20:40:18 Lena Voytek nominated for series Ubuntu Kinetic
2022-09-29 20:40:18 Lena Voytek bug task added dnsmasq (Ubuntu Kinetic)
2022-09-29 20:40:18 Lena Voytek nominated for series Ubuntu Jammy
2022-09-29 20:40:18 Lena Voytek bug task added dnsmasq (Ubuntu Jammy)
2022-09-29 20:40:24 Lena Voytek dnsmasq (Ubuntu Jammy): status New Confirmed
2022-09-29 20:40:26 Lena Voytek dnsmasq (Ubuntu Kinetic): status New Confirmed
2022-09-30 00:27:29 Ubuntu Foundations Team Bug Bot tags patch
2022-09-30 00:27:38 Ubuntu Foundations Team Bug Bot bug added subscriber Ubuntu Review Team
2022-09-30 03:20:28 Christian Ehrhardt  dnsmasq (Ubuntu Kinetic): assignee Lena Voytek (lvoytek)
2022-09-30 03:20:34 Christian Ehrhardt  dnsmasq (Ubuntu Jammy): assignee Lena Voytek (lvoytek)
2022-09-30 03:20:42 Christian Ehrhardt  tags patch patch server-todo
2022-09-30 16:27:08 Lena Voytek dnsmasq (Ubuntu Kinetic): status Confirmed In Progress
2022-10-05 15:12:29 Lena Voytek bug added subscriber Ubuntu Server
2022-10-06 22:48:26 Launchpad Janitor merge proposal linked https://code.launchpad.net/~lvoytek/ubuntu/+source/dnsmasq/+git/dnsmasq/+merge/431166
2022-10-08 14:15:57 Launchpad Janitor dnsmasq (Ubuntu Kinetic): status In Progress Fix Released
2022-10-14 21:36:41 Lena Voytek description Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1 [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] To test the error with lxd, run the following: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install dnsutils dnsmasq -y # systemctl enable dnsmasq Access internet through dnsmasq, flaky internet can be modeled by disconnecting from router, etc. This will lead to denial of DNS query retries. [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1
2022-10-14 21:36:47 Lena Voytek dnsmasq (Ubuntu Jammy): status Confirmed In Progress
2022-10-14 21:46:34 Launchpad Janitor merge proposal linked https://code.launchpad.net/~lvoytek/ubuntu/+source/dnsmasq/+git/dnsmasq/+merge/431606
2022-10-17 22:38:37 Launchpad Janitor merge proposal unlinked https://code.launchpad.net/~lvoytek/ubuntu/+source/dnsmasq/+git/dnsmasq/+merge/431606
2022-10-17 22:51:52 Lena Voytek merge proposal linked https://code.launchpad.net/~lvoytek/ubuntu/+source/dnsmasq/+git/dnsmasq/+merge/431606
2022-12-06 17:34:14 Lena Voytek description [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] To test the error with lxd, run the following: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install dnsutils dnsmasq -y # systemctl enable dnsmasq Access internet through dnsmasq, flaky internet can be modeled by disconnecting from router, etc. This will lead to denial of DNS query retries. [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1 [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] The fix can be tested using 2 lxd containers, 1 for running the fix, and 1 for acting as a dns server. Start by setting up the dns server container: # lxc launch images:ubuntu/jammy dns-resolver # lxc exec dns-resolver bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install net-tools dnsmasq -y # systemctl enable dnsmasq Get the container's ip on lxd's network, in this case ifconfig is used, showing 10.62.42.157: # ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.62.42.157 netmask 255.255.255.0 broadcast 10.62.42.255 ... Now set up the test container: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf Use other container's ip here # echo "nameserver 10.62.42.157 nameserver 127.0.0.1" | tee /etc/resolv.conf # apt install dnsmasq -y # systemctl enable dnsmasq Setup bind9 to delay ping resolution failure when testing # apt install bind9 -y # cat <<EOF >/etc/bind/named.conf.options acl goodclients { localhost; }; options { directory "/var/cache/bind"; dnssec-validation auto; listen-on-v6 { any; }; recursion yes; allow-query { goodclients; }; forwarders { 8.8.8.8; }; }; EOF # service bind9 restart On the dns server side, set the nameserver to 127.0.0.1 to cause denials on the test server: # echo nameserver 127.0.0.1 | tee /etc/resolv.conf; systemctl restart dnsmasq Now ping a known domain on the test container, and while it runs set the dns server side nameserver back to 8.8.8.8: # ping ubuntu.com > swap containers # echo nameserver 8.8.8.8 | tee /etc/resolv.conf; systemctl restart dnsmasq Ping will continue to not pick up the domain and fails with: ping: ubuntu.com: Temporary failure in name resolution With the fix, ping should now pick up the new successful responses: PING ubuntu.com (185.125.190.29) 56(84) bytes of data. 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=1 ttl=48 time=165 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=2 ttl=48 time=162 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=3 ttl=48 time=166 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=4 ttl=48 time=164 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=5 ttl=48 time=163 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=6 ttl=48 time=163 ms ^C --- ubuntu.com ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5005ms [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1
2023-01-04 18:38:24 Lena Voytek description [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] The fix can be tested using 2 lxd containers, 1 for running the fix, and 1 for acting as a dns server. Start by setting up the dns server container: # lxc launch images:ubuntu/jammy dns-resolver # lxc exec dns-resolver bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install net-tools dnsmasq -y # systemctl enable dnsmasq Get the container's ip on lxd's network, in this case ifconfig is used, showing 10.62.42.157: # ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.62.42.157 netmask 255.255.255.0 broadcast 10.62.42.255 ... Now set up the test container: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf Use other container's ip here # echo "nameserver 10.62.42.157 nameserver 127.0.0.1" | tee /etc/resolv.conf # apt install dnsmasq -y # systemctl enable dnsmasq Setup bind9 to delay ping resolution failure when testing # apt install bind9 -y # cat <<EOF >/etc/bind/named.conf.options acl goodclients { localhost; }; options { directory "/var/cache/bind"; dnssec-validation auto; listen-on-v6 { any; }; recursion yes; allow-query { goodclients; }; forwarders { 8.8.8.8; }; }; EOF # service bind9 restart On the dns server side, set the nameserver to 127.0.0.1 to cause denials on the test server: # echo nameserver 127.0.0.1 | tee /etc/resolv.conf; systemctl restart dnsmasq Now ping a known domain on the test container, and while it runs set the dns server side nameserver back to 8.8.8.8: # ping ubuntu.com > swap containers # echo nameserver 8.8.8.8 | tee /etc/resolv.conf; systemctl restart dnsmasq Ping will continue to not pick up the domain and fails with: ping: ubuntu.com: Temporary failure in name resolution With the fix, ping should now pick up the new successful responses: PING ubuntu.com (185.125.190.29) 56(84) bytes of data. 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=1 ttl=48 time=165 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=2 ttl=48 time=162 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=3 ttl=48 time=166 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=4 ttl=48 time=164 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=5 ttl=48 time=163 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=6 ttl=48 time=163 ms ^C --- ubuntu.com ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5005ms [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1 [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] The fix can be tested using 2 lxd containers, 1 for running the fix, and 1 for acting as a dns server. Start by setting up the dns server container: # lxc launch images:ubuntu/jammy dns-resolver # lxc exec dns-resolver bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install net-tools dnsmasq -y # systemctl enable dnsmasq Get the container's ip on lxd's network, in this case ifconfig is used, showing 10.62.42.157: # ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500         inet 10.62.42.157 netmask 255.255.255.0 broadcast 10.62.42.255         ... Now set up the test container: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf Use other container's ip here # echo "nameserver 10.62.42.157 nameserver 127.0.0.1" | tee /etc/resolv.conf # apt install dnsmasq -y # systemctl enable dnsmasq Setup bind9 to delay ping resolution failure when testing # apt install bind9 -y # cat <<EOF >/etc/bind/named.conf.options acl goodclients {         localhost; }; options {         directory "/var/cache/bind";         dnssec-validation auto;         listen-on-v6 { any; };         recursion yes;         allow-query { goodclients; };         forwarders {                 8.8.8.8;         }; }; EOF # service bind9 restart On the dns server side, set the nameserver to 127.0.0.1 to cause denials on the test server: # echo nameserver 127.0.0.1 | tee /etc/resolv.conf; systemctl restart dnsmasq Now ping a known domain on the test container, and while it runs set the dns server side nameserver back to 8.8.8.8: # ping ubuntu.com > swap containers # echo nameserver 8.8.8.8 | tee /etc/resolv.conf; systemctl restart dnsmasq Ping will continue to not pick up the domain and fails with: ping: ubuntu.com: Temporary failure in name resolution With the fix, ping should now pick up the new successful responses: PING ubuntu.com (185.125.190.29) 56(84) bytes of data. 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=1 ttl=48 time=165 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=2 ttl=48 time=162 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=3 ttl=48 time=166 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=4 ttl=48 time=164 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=5 ttl=48 time=163 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=6 ttl=48 time=163 ms ^C --- ubuntu.com ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5005ms Note: ping is used here instead of another dns tester such as dig because it continues its dns resolution attempts with retry packets even after receiving a REFUSED error. Other programs fail immediately on REFUSED and are unable to send duplicate packets to reproduce the issue. [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1
2023-01-18 20:27:07 Lena Voytek description [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] The fix can be tested using 2 lxd containers, 1 for running the fix, and 1 for acting as a dns server. Start by setting up the dns server container: # lxc launch images:ubuntu/jammy dns-resolver # lxc exec dns-resolver bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install net-tools dnsmasq -y # systemctl enable dnsmasq Get the container's ip on lxd's network, in this case ifconfig is used, showing 10.62.42.157: # ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500         inet 10.62.42.157 netmask 255.255.255.0 broadcast 10.62.42.255         ... Now set up the test container: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf Use other container's ip here # echo "nameserver 10.62.42.157 nameserver 127.0.0.1" | tee /etc/resolv.conf # apt install dnsmasq -y # systemctl enable dnsmasq Setup bind9 to delay ping resolution failure when testing # apt install bind9 -y # cat <<EOF >/etc/bind/named.conf.options acl goodclients {         localhost; }; options {         directory "/var/cache/bind";         dnssec-validation auto;         listen-on-v6 { any; };         recursion yes;         allow-query { goodclients; };         forwarders {                 8.8.8.8;         }; }; EOF # service bind9 restart On the dns server side, set the nameserver to 127.0.0.1 to cause denials on the test server: # echo nameserver 127.0.0.1 | tee /etc/resolv.conf; systemctl restart dnsmasq Now ping a known domain on the test container, and while it runs set the dns server side nameserver back to 8.8.8.8: # ping ubuntu.com > swap containers # echo nameserver 8.8.8.8 | tee /etc/resolv.conf; systemctl restart dnsmasq Ping will continue to not pick up the domain and fails with: ping: ubuntu.com: Temporary failure in name resolution With the fix, ping should now pick up the new successful responses: PING ubuntu.com (185.125.190.29) 56(84) bytes of data. 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=1 ttl=48 time=165 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=2 ttl=48 time=162 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=3 ttl=48 time=166 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=4 ttl=48 time=164 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=5 ttl=48 time=163 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=6 ttl=48 time=163 ms ^C --- ubuntu.com ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5005ms Note: ping is used here instead of another dns tester such as dig because it continues its dns resolution attempts with retry packets even after receiving a REFUSED error. Other programs fail immediately on REFUSED and are unable to send duplicate packets to reproduce the issue. [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1 [Impact] When a DNS query fails to complete and the system retries it, subsequent copies of the query will be refused by dnsmasq. The client will automatically receive the REFUSED return value without a retry attempt. Adding this fix will stop dnsmasq from unnecessarily breaking connections, especially for situations where an internet connection is flaky. This bug is fixed by patching in an upstream commit - https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be - which allows retried DNS requests through rather than refusing them. [Test Plan] The fix can be tested using 2 lxd containers, 1 for running the fix, and 1 for acting as a dns server. Start by setting up the dns server container: # lxc launch images:ubuntu/jammy dns-resolver # lxc exec dns-resolver bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf # echo nameserver 8.8.8.8 | tee /etc/resolv.conf # apt install net-tools dnsmasq -y # systemctl enable dnsmasq Get the container's ip on lxd's network, in this case ifconfig is used, showing 10.62.42.157: # ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500         inet 10.62.42.157 netmask 255.255.255.0 broadcast 10.62.42.255         ... Now set up the test container: # lxc launch images:ubuntu/jammy test-dnsmasq # lxc exec test-dnsmasq bash # apt update && apt dist-upgrade -y # systemctl disable systemd-resolved # systemctl stop systemd-resolved # unlink /etc/resolv.conf Use other container's ip here, along with an ip that does not resolve # echo "nameserver 10.62.42.157 nameserver 192.0.2.1" | tee /etc/resolv.conf # apt install dnsmasq -y # systemctl enable dnsmasq On the dns server side, set the nameserver to 127.0.0.1 to cause denials on the test server: # echo nameserver 127.0.0.1 | tee /etc/resolv.conf; systemctl restart dnsmasq Now ping a known domain on the test container, and while it runs set the dns server side nameserver back to 8.8.8.8: # ping ubuntu.com > swap containers # echo nameserver 8.8.8.8 | tee /etc/resolv.conf; systemctl restart dnsmasq Ping will continue to not pick up the domain and fails with: ping: ubuntu.com: Temporary failure in name resolution With the fix, ping should now pick up the new successful responses: PING ubuntu.com (185.125.190.29) 56(84) bytes of data. 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=1 ttl=48 time=165 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=2 ttl=48 time=162 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=3 ttl=48 time=166 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=4 ttl=48 time=164 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=5 ttl=48 time=163 ms 64 bytes from website-content-cache-3.ps5.canonical.com (185.125.190.29): icmp_seq=6 ttl=48 time=163 ms ^C --- ubuntu.com ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5005ms Note: ping is used here instead of another dns tester such as dig because it continues its dns resolution attempts with retry packets even after receiving a REFUSED error. Other programs fail immediately on REFUSED and are unable to send duplicate packets to reproduce the issue. [Where problems could occur] This change was added upstream in version 2.87, which means it has not been tested in many situations alongside 2.86. Allowing the retries could lead to a flood of requests to remote DNS servers if the replies are unable to make it back through dnsmasq to the user. [Other Info] This bug was fixed in Kinetic in version 2.86-1.1ubuntu2. [Original Description] Duplicate or retried DNS queries will return REFUSED for one of the queries causing intermittent failures in clients. This probably breaks lots of things, but for me is causing 22.04's internet connection sharing to be unstable. It's particularly bad for my Xbox which seems to like sending duplicate queries. Here's an example capture: 22:37:25.308212 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332711 IP 10.42.0.16.54248 > 10.42.0.1.53: 22442+ A? title.auth.xboxlive.com. (41) 22:37:25.332740 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 Refused 0/0/0 (41) 22:37:25.353003 IP 10.42.0.1.53 > 10.42.0.16.54248: 22442 2/0/0 CNAME title.auth.xboxlive.com.akadns.net., A 40.64.90.82 (105) This has been fixed in upstream as of Sept 2021 in the unreleased 2.87 version. It's apparently a regression in version 2.86 (also released in Sept 2021). Ubuntu 22.04 and later all use the broken 2.86 version. Upstream fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=2561f9fe0eb9c0be1df48da1e2bd3d3feaa138c2 Upstream bug thread: https://www.mail-archive.com/search?l=dnsmasq-discuss%40lists.thekelleys.org.uk&q=subject:%22%5C%5BDnsmasq%5C-discuss%5C%5D+REFUSED+after+dropped+packets%22&o=oldest&f=1
2023-02-16 19:53:43 Andreas Hasenack dnsmasq (Ubuntu Jammy): status In Progress Fix Committed
2023-02-16 19:53:46 Andreas Hasenack bug added subscriber Ubuntu Stable Release Updates Team
2023-02-16 19:53:47 Andreas Hasenack bug added subscriber SRU Verification
2023-02-16 19:53:51 Andreas Hasenack tags patch server-todo patch server-todo verification-needed verification-needed-jammy
2023-02-21 07:18:51 Reuben Lifshay tags patch server-todo verification-needed verification-needed-jammy patch server-todo verification-done-jammy verification-needed
2023-02-22 22:12:12 Lena Voytek tags patch server-todo verification-done-jammy verification-needed patch server-todo verification-done verification-done-jammy
2023-03-29 00:51:33 Launchpad Janitor dnsmasq (Ubuntu Jammy): status Fix Committed Fix Released
2023-03-29 00:51:38 Chris Halse Rogers removed subscriber Ubuntu Stable Release Updates Team