jammy's version breaks existing dhcp scripts with relay

[Timo M]

[ Impact ]

When upgrading from focal to jammy, existing dnsmasq dhcp-scripts stopped working in an environment where a DHCP relay is in use. Instead of the expected client IP address, the script gets the _relay_ IP address as an argument.

This was fixed in 2.87, therefore making only jammy carry an affected package.

[ Test Plan ]

To easily test this on a single machine, a test script is being provided to setup networking and dnsmasq configuration.

# Launch a jammy VM

lxc launch ubuntu-daily:jammy j-dnsmasq-2042587 --vm

# open a root shell in that VM. All subsequent commands must be executed as root in that VM

lxc shell j-dnsmasq-2042587

# download test script

wget https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/2042587/+attachment/5738174/+files/setup-and-server.sh

# make it executable

chmod +x setup-and-server.sh

# install dnsmasq. Ignore the postinst error (because systemd-resolved is also running and there is a port conflict)

apt update && apt install dnsmasq -y

# run the setup script. It will configure things and start dnsmasq ready to be tested

./setup-and-server.sh

# in another root session inside the vm (so run "lxc shell j-dnsmasq-2042587" in another terminal), run the proposed commands from the setup script (and press ctrl-c after the result is shown):

No DHCP relay:

  ip netns exec client dhclient -d -v p2

The setup script should log an IP that is not a relay. For example:
dnsmasq-dhcp: DHCPDISCOVER(p1) aa:a0:9d:00:5b:d6
dnsmasq-dhcp: DHCPOFFER(p1) 192.168.47.150 aa:a0:9d:00:5b:d6
dnsmasq-dhcp: DHCPREQUEST(p1) 192.168.47.150 aa:a0:9d:00:5b:d6
dnsmasq-dhcp: DHCPACK(p1) 192.168.47.150 aa:a0:9d:00:5b:d6 j-dnsmasq-2042587
###########################
IP = 192.168.47.150
###########################

With DHCP relay set to 192.168.47.9, IP should NOT be that address:

  ip netns exec client dhclient -d -v p2 -g 192.168.47.9

With the affected dnsmasq package, we will see an error:
dnsmasq-dhcp: DHCPREQUEST(p1) 192.168.47.150 aa:a0:9d:00:5b:d6
dnsmasq-dhcp: DHCPACK(p1) 192.168.47.150 aa:a0:9d:00:5b:d6 j-dnsmasq-2042587
###########################
IP = 192.168.47.9
TEST FAILED
###########################

The error is that the obtained IP is that of the dhcp relay (provided via the -g option).

With the fixed dnsmasq package, "TEST FAILED" must not appear, and the IP should not be that of the provided dhcp relay.

[ Where problems could occur ]

If the fix is incorrect, it would mean the dhcp-script would get an incorrect IP again, or perhaps we could have crashes in dnsmasq when dealing with buffers and pointers if the dhcp-script option is in use.
This fix was committed upstream a few months after the bug was introduced, so it took a while to be noticed.

[ Other Info ]
Not at this time.

[ Original description ]

When upgrading from focal to jammy, existing dnsmasq dhcp-scripts stopped working in an environment where a DHCP relay is in use. Instead of the expected client IP address, the script gets the _relay_ IP address as an argument. From dnsmasq documentation for --dhcp-script:

> The arguments to the process are "add", "old" or "del", the MAC address of the host (or DUID for IPv6) , the IP address, and the hostname, if known.

I believe the change has been inadverently made in upstream commit 527c3c7d0d3bb4bf5fad699f10cf0d1a45a54692 (https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=blobdiff;f=src/helper.c;h=02340a01c00031db0cc682c8a4a279cfc1db574e;hp=d81de9622e6d484a264496b2cd3638b4e15e9677;hb=527c3c7d0d3bb4bf5fad699f10cf0d1a45a54692;hpb=fcb4dcaf7cc8a86ac2533b933161b6455f75bf8f) as the commit message only speaks about inet_ntoa replacement and not the behavioral change it also introduces (previously the relay address was only set to the environment variable, now it effectively overrides the prevoiusly set client's IP address).

dnsmasq 2.86-1.1ubuntu0.3 / Ubuntu 22.04

[Mitchell Dzurick (mitchdz)]

Thanks for taking the time to report this bug Timo!

Do you have steps for how you setup dnsmasq so it can be more easy to reproduce your issue?

[Timo M (timotunk)]

I'm so sorry this is so complex, but I couldn't figure out any simpler way to reproduce as they components involved do network configuration.

Anyway, please find attached a docker compose file and related dnsmasq configurations (mapped automatically in that compose file) that helps in the reproduction by setting up two different dhcp servers based on dnsmasq (focal and jammy) and a machine that can be used to request a dhcp address.

To reproduce the bug:
1) first install everything needed by running
   docker-compose up -d
2) wait until everything has been installed, check for example that the output of has something from all 3 containers and is not getting any more lines
   docker-compose logs --follow
3) stop jammy version of the server with
   docker-compose stop jammy-server
4) get inside client machine to run dhclient. Run it first without fake relay address, then with one (to trigger the DHCP relay behaviour in dnsmasq)
   docker-compose exec client bash
   dhclient -d -v
   (ctrl-c after a while)
   dhclient -d -v -g 192.168.0.10
   (ctrl-c after a while)
5) observe server's log
dnsmasq-script-sample-focal-server-1 | dnsmasq-dhcp: DHCPREQUEST(eth0) 192.168.0.3 02:42:c0:a8:00:04
dnsmasq-script-sample-focal-server-1 | dnsmasq-dhcp: DHCPACK(eth0) 192.168.0.3 02:42:c0:a8:00:04 5a49beb3cd53
dnsmasq-script-sample-focal-server-1 | hello from dhcp hook script, IP argument is 192.168.0.3
dnsmasq-script-sample-focal-server-1 | dnsmasq-dhcp: DHCPREQUEST(eth0) 192.168.0.3 02:42:c0:a8:00:04
dnsmasq-script-sample-focal-server-1 | dnsmasq-dhcp: DHCPACK(eth0) 192.168.0.3 02:42:c0:a8:00:04 5a49beb3cd53
dnsmasq-script-sample-focal-server-1 | hello from dhcp hook script, IP argument is 192.168.0.3
6) now stop focal version of server and start jammy:
   docker-compose stop focal-server
   docker-compose start jammy-server
7) repeat steps 5 and 6, this time the output has the wroing ip in the 2nd hook script invocation
dnsmasq-script-sample-jammy-server-1 | dnsmasq-dhcp: DHCPREQUEST(eth0) 192.168.0.1 02:42:c0:a8:00:04
dnsmasq-script-sample-jammy-server-1 | dnsmasq-dhcp: DHCPACK(eth0) 192.168.0.1 02:42:c0:a8:00:04 5a49beb3cd53
dnsmasq-script-sample-jammy-server-1 | hello from dhcp hook script, IP argument is 192.168.0.1
dnsmasq-script-sample-jammy-server-1 | dnsmasq-dhcp: DHCPREQUEST(eth0) 192.168.0.1 02:42:c0:a8:00:04
dnsmasq-script-sample-jammy-server-1 | dnsmasq-dhcp: DHCPACK(eth0) 192.168.0.1 02:42:c0:a8:00:04 5a49beb3cd53
dnsmasq-script-sample-jammy-server-1 | hello from dhcp hook script, IP argument is 192.168.0.10

[Timo M (timotunk)]

..also, apologies for all the typos I managed to place there. Step 7 was meant to say "repeat steps 4 and 5"

[Lucas Kanashiro (lucaskanashiro)]

Hi Timo,

Thanks for the reproduction steps. I just tried them out locally and I was able to reproduce the described behavior. I am marking this as triaged and subscribing ubuntu-server.

[Andreas Hasenack (ahasenack)]

I believe this to be the fix: https://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=patch;h=1c8855ed10d3923a9a4fd8a89f1c95439d4c8827

From 1c8855ed10d3923a9a4fd8a89f1c95439d4c8827 Mon Sep 17 00:00:00 2001
From: Simon Kelley <email address hidden>
Date: Sun, 26 Dec 2021 16:35:54 +0000
Subject: [PATCH] Fix wrong client address for dhcp-script when DHCPv4 relay in
 use.

---
 src/helper.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/helper.c b/src/helper.c
index 02340a0..455a68c 100644
--- a/src/helper.c
+++ b/src/helper.c
@@ -432,8 +432,8 @@ int create_helper(int event_fd, int err_fd, uid_t uid, gid_t gid, long max_fd)
   buf = grab_extradata_lua(buf, end, "relay_address");
        else if (data.giaddr.s_addr != 0)
   {
- inet_ntop(AF_INET, &data.giaddr, daemon->addrbuff, ADDRSTRLEN);
- lua_pushstring(lua, daemon->addrbuff);
+ inet_ntop(AF_INET, &data.giaddr, daemon->dhcp_buff2, ADDRSTRLEN);
+ lua_pushstring(lua, daemon->dhcp_buff2);
     lua_setfield(lua, -2, "relay_address");
   }

@@ -615,7 +615,7 @@ int create_helper(int event_fd, int err_fd, uid_t uid, gid_t gid, long max_fd)
      {
        const char *giaddr = NULL;
        if (data.giaddr.s_addr != 0)
- giaddr = inet_ntop(AF_INET, &data.giaddr, daemon->addrbuff, ADDRSTRLEN);
+ giaddr = inet_ntop(AF_INET, &data.giaddr, daemon->dhcp_buff2, ADDRSTRLEN);
        my_setenv("DNSMASQ_RELAY_ADDRESS", giaddr, &err);
      }

--
2.20.1

Committed a few months later in the same area. Will try to test it.

If that's really the fix, then only jammy is affected, because the patch is in v2.87 and later:
 dnsmasq | 2.80-1.1ubuntu1 | focal | source
 dnsmasq | 2.80-1.1ubuntu1.7 | focal-security | source
 dnsmasq | 2.80-1.1ubuntu1.7 | focal-updates | source
 dnsmasq | 2.86-1.1 | jammy | source
 dnsmasq | 2.86-1.1ubuntu0.3 | jammy-security | source
 dnsmasq | 2.86-1.1ubuntu0.4 | jammy-proposed | source
 dnsmasq | 2.86-1.1ubuntu0.4 | jammy-updates | source
 dnsmasq | 2.89-1 | lunar | source
 dnsmasq | 2.89-1 | mantic | source
 dnsmasq | 2.89-1 | noble | source

[Andreas Hasenack (ahasenack)]

Reproduced the problem and verified the fix.

@timotunk, if you would like to give the fix a try, it's available in this PPA for jammy:

https://launchpad.net/~ahasenack/+archive/ubuntu/dnsmasq-relay-2042587/+packages

I'll in the meantime prepare the SRU paperwork and merge proposal for review.

[Andreas Hasenack (ahasenack)]

Test script

[Steve Langasek (vorlon)]

Hello Timo, or anyone else affected,

Accepted dnsmasq into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dnsmasq/2.86-1.1ubuntu0.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo M (timotunk)]

Hello,

and thank you, seems to fix the issue. I updated my jammy's to dsnmasq version 2.86-1.1ubuntu0.5, and tested with the same setup as I reported the original bug. Can confirm that the DHCP hook script now works exatly like it used to in focal, getting the correct IP address as an argument.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted dnsmasq (2.86-1.1ubuntu0.5) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

systemd/249.11-0ubuntu3.11 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#dnsmasq

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Andreas Hasenack (ahasenack)]

The autopkgtests are green now.

[Andreas Hasenack (ahasenack)]

Reproducing the bug

root@j-dnsmasq-2042587:~# apt-cache policy dnsmasq
dnsmasq:
  Installed: 2.86-1.1ubuntu0.4
  Candidate: 2.86-1.1ubuntu0.4
  Version table:
 *** 2.86-1.1ubuntu0.4 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages
        100 /var/lib/dpkg/status
     2.86-1.1ubuntu0.3 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages
     2.86-1.1 500
        500 http://br.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages

Running the setup script:
# ./setup-and-server.sh

Cleaning up
Setting things up...

Commands to run in another terminal:

No DHCP relay:

  ip netns exec client dhclient -d -v p2

With DHCP relay set to 192.168.47.9, IP should NOT be that address:

  ip netns exec client dhclient -d -v p2 -g 192.168.47.9

Starting server, observe output and run the test commands above.
Press ctrl-c when done.

dnsmasq: started, version 2.86 cachesize 150
dnsmasq: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
dnsmasq-dhcp: DHCP, IP range 192.168.47.10 -- 192.168.47.250, lease time 12h
dnsmasq-dhcp: DHCP, sockets bound exclusively to interface p1
dnsmasq: reading /etc/resolv.conf
dnsmasq: using nameserver 127.0.0.53#53
dnsmasq: read /etc/hosts - 7 addresses

For the "no DHCP relay" case:
# ip netns exec client dhclient -d -v p2
(...)
bound to 192.168.47.209 -- renewal in 21350 seconds.

And the server logs:
dnsmasq-dhcp: DHCPDISCOVER(p1) ee:a3:dd:58:d8:35
dnsmasq-dhcp: DHCPOFFER(p1) 192.168.47.209 ee:a3:dd:58:d8:35
dnsmasq-dhcp: DHCPREQUEST(p1) 192.168.47.209 ee:a3:dd:58:d8:35
dnsmasq-dhcp: DHCPACK(p1) 192.168.47.209 ee:a3:dd:58:d8:35 j-dnsmasq-2042587
###########################
IP = 192.168.47.209
###########################

That's the correct IP we want to see. Now, on to the bug, which is WITH a DHCP relay.

We run this on the client:
# ip netns exec client dhclient -d -v p2 -g 192.168.47.9
(...)
DHCPREQUEST for 192.168.47.209 on p2 to 255.255.255.255 port 67 (xid=0x7f70085a)

And the server logs the incorrect iP:
dnsmasq-dhcp: DHCPREQUEST(p1) 192.168.47.209 ee:a3:dd:58:d8:35
dnsmasq-dhcp: DHCPACK(p1) 192.168.47.209 ee:a3:dd:58:d8:35 j-dnsmasq-2042587
###########################
IP = 192.168.47.9
TEST FAILED
###########################

Now let's use the package from proposed:

root@j-dnsmasq-2042587:~# apt-cache policy dnsmasq
dnsmasq:
  Installed: 2.86-1.1ubuntu0.5
  Candidate: 2.86-1.1ubuntu0.5
  Version table:
 *** 2.86-1.1ubuntu0.5 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     2.86-1.1ubuntu0.4 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages
     2.86-1.1ubuntu0.3 500
        500 http://br.archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages
     2.86-1.1 500
        500 http://br.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages

We run the server script again:

The no-relay case:
# ip netns exec client dhclient -d -v p2
(...)
bound to 192.168.47.155 -- renewal in 19424 seconds.

Where th...

[Launchpad Janitor (janitor)]

This bug was fixed in the package dnsmasq - 2.86-1.1ubuntu0.5

---------------
dnsmasq (2.86-1.1ubuntu0.5) jammy; urgency=medium

  * src/dnsmasq.c: Fix a crash that can happen when an empty resolv.conf is
    reloaded (LP: #2045570)
  * src/helper.c: Fix wrong client address for dhcp-script when DHCPv4 relay
    in use (LP: #2042587)

 -- Andreas Hasenack <email address hidden> Thu, 11 Jan 2024 09:21:27 -0300

[Robie Basak (racb)]

The verification of the Stable Release Update for dnsmasq has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.