[PATCH] systemd-resolved can't follow more than 8 CNAMEs

[Vincent Renardias]

[Impact]

Using systemd-resolved to resolve a hostname which has more than 8 CNAME redirects will fail because of the hard-coded limit. While this case is somewhat rare, the original reporter demonstrated a real-world scenario where this happened (although that particular hostname seems to be fixed now).

[Test Plan]

This test plan uses a LXC container to test systemd-resolved on Focal. If LXD has not been configured on your system, start with:

$ lxd init --auto

Then, create a Focal container with:

$ lxc launch ubuntu-daily:focal focal

Install dnsmasq-base if needed:

$ apt install dnsmasq-base

Stop other DNS servers:

$ systemctl stop systemd-resolved
$ kill -9 $(pgrep dnsmasq)

Now, on the host start a new DNS server that listens on lxdbr0, and sets up an A record, and many CNAME records which ultimately redirect to the A record:

$ dnsmasq \
--cname=test10.lan,test9.lan \
--cname=test9.lan,test8.lan \
--cname=test8.lan,test7.lan \
--cname=test7.lan,test6.lan \
--cname=test6.lan,test5.lan \
--cname=test5.lan,test4.lan \
--cname=test4.lan,test3.lan \
--cname=test3.lan,test2.lan \
--cname=test2.lan,test1.lan \
--cname=test1.lan,test0.lan \
-k -i lxdbr0 -z -I lo --host-record=test0.lan,$IP

where $IP is any host on your network.

Now, obtain a shell in the Focal container:

$ lxc exec focal bash

Attempt to resolve test10.lan:

$ resolvectl query test10.lan
test10.lan: resolve call failed: CNAME loop detected, or CNAME resolving disabled on 'test2.lan'

On an affected system, the above error will be seen. On a patched system, the hostname should be resolved.

[Where problems could occur]

The patch simply increases the maximum CNAME redirects that are allowed from 8 to 16, so a reasonable limit is still imposed. If an application specifically relied on systemd-resolved's limit being at 8, then that application would potentially see new behavior.

[Original Description]

On Ubuntu 20.04 (systemd v245.4-4ubuntu3.21), hostname resolution only follows 8 CNAME redirections maximum.

So when using a service like Azure Virtual Desktop that has between 9 and 12 redirections, name resolution fails.

$ host client.wvd.microsoft.com
Host client.wvd.microsoft.com not found: 2(SERVFAIL)
$ resolvectl query client.wvd.microsoft.com
client.wvd.microsoft.com: resolve call failed: CNAME loop detected, or CNAME resolving disabled on 'waws-prod-zrh-ff7172dd.sip.p.azurewebsites.windows.net'

On the other hand it's working fine on Ubuntu 20.04 because CNAME loop limit has been raised from 8 to 16.

$ host client.wvd.microsoft.com
client.wvd.microsoft.com is an alias for client.privatelink-global.wvd.microsoft.com.
client.privatelink-global.wvd.microsoft.com is an alias for client.privatelink.wvd.microsoft.com.
client.privatelink.wvd.microsoft.com is an alias for rdweb.wvd.microsoft.com.
rdweb.wvd.microsoft.com is an alias for rdweb.privatelink-global.wvd.microsoft.com.
rdweb.privatelink-global.wvd.microsoft.com is an alias for rdweb.privatelink.wvd.microsoft.com.
rdweb.privatelink.wvd.microsoft.com is an alias for rdweb-prod-geo.trafficmanager.net.
rdweb-prod-geo.trafficmanager.net is an alias for mrs-chnor1c101-rdweb-prod.wvd-ase-chnor1c101-prod.p.azurewebsites.net.
mrs-chnor1c101-rdweb-prod.wvd-ase-chnor1c101-prod.p.azurewebsites.net is an alias for waws-prod-zrh-63daa049.sip.p.azurewebsites.windows.net.
waws-prod-zrh-63daa049.sip.p.azurewebsites.windows.net is an alias for waws-prod-zrh-63daa049.cloudapp.net.
waws-prod-zrh-63daa049.cloudapp.net has address 51.107.69.35

Here's a quick fix that raises the max CNAME limit from 8 to 16 like it is in Ubuntu 22.04, it fixes the problem for me.

Best regards,
Vincent.

--- systemd-245.4.ORIG/src/resolve/resolved-dns-query.c 2023-06-15 16:58:25.454156663 +0200
+++ systemd-245.4/src/resolve/resolved-dns-query.c 2023-06-01 14:30:09.000000000 +0200
@@ -10,7 +10,7 @@
 #include "resolved-etc-hosts.h"
 #include "string-util.h"

-#define CNAME_MAX 8
+#define CNAME_MAX 16
 #define QUERIES_MAX 2048
 #define AUXILIARY_QUERIES_MAX 64

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: systemd 249.11-0ubuntu3.9 [modified: usr/lib/sysctl.d/50-default.conf]
ProcVersionSignature: Ubuntu 5.19.0-42.43~22.04.1-generic 5.19.17
Uname: Linux 5.19.0-42-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: ubuntu:GNOME
Date: Thu Jun 15 16:53:19 2023
InstallationDate: Installed on 2022-11-22 (205 days ago)
InstallationMedia: Error: [Errno 13] Permission denied: '/var/log/installer/media-info'
MachineType: Dell Inc. Latitude 5591
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.19.0-42-generic root=/dev/mapper/AmadeUbuntu22-root ro apparmor=1 security=apparmor audit=1 acpi_rev_override load_nvme=YES nouveau.modeset=0 dis_ucode_ldr quiet splash
RebootRequiredPkgs: Error: path contained symlinks.
SourcePackage: systemd
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 11/21/2022
dmi.bios.release: 1.25
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 1.25.0
dmi.board.name: 0DVVG1
dmi.board.vendor: Dell Inc.
dmi.board.version: A00
dmi.chassis.type: 10
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr1.25.0:bd11/21/2022:br1.25:svnDellInc.:pnLatitude5591:pvr:rvnDellInc.:rn0DVVG1:rvrA00:cvnDellInc.:ct10:cvr:sku0819:
dmi.product.family: Latitude
dmi.product.name: Latitude 5591
dmi.product.sku: 0819
dmi.sys.vendor: Dell Inc.
modified.conffile..etc.cron.daily.apport: [deleted]
mtime.conffile..etc.systemd.journald.conf: 2022-11-22T13:40:37.558934

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Nick Rosbrook (enr0n)]

I confirmed this in a Focal container:

root@focal:~# resolvectl query client.wvd.microsoft.com
client.wvd.microsoft.com: resolve call failed: CNAME loop detected, or CNAME resolving disabled on 'waws-prod-yq1-4474575f.sip.p.azurewebsites.windows.net'

This limit has in fact been increased to 16 upstream, which is the case for Jammy and newer (https://github.com/systemd/systemd/blob/34c4496ef2711d2a924e6f88fe3ff31cda080115/src/resolve/resolved-dns-query.c#LL17C12-L17C12).

I don't think this will be prioritized, but I don't see why it can't be fixed in a future SRU.

[Brian Murray (brian-murray)]

Hello Vincent, or anyone else affected,

Accepted systemd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/245.4-4ubuntu3.23 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (245.4-4ubuntu3.23) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

casync/2+20190213-1 (armhf)
gvfs/1.44.1-1ubuntu1.2 (ppc64el)
linux-gcp-5.15/5.15.0-1048.56~20.04.1 (arm64)
linux-hwe-5.15/5.15.0-91.101~20.04.1 (armhf)
linux-oracle-5.15/5.15.0-1049.55~20.04.1 (arm64)
mariadb-10.3/1:10.3.38-0ubuntu0.20.04.1 (armhf)
netplan.io/0.104-0ubuntu2~20.04.4 (s390x)
puppet/5.5.10-4ubuntu3 (armhf)
upower/0.99.11-1build2 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Nick Rosbrook (enr0n)]

I have verified the fix using systemd 245.4-4ubuntu3.23 from focal-proposed.

On the host, I have a Focal container, and killed the existing DNS servers:

nr@clean-jammy-amd64:~$ lxc list
+-------+---------+------+----------------------------------------------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+------+----------------------------------------------+-----------+-----------+
| focal | RUNNING | | fd42:b04:cc58:1a82:216:3eff:fe28:d8d9 (eth0) | CONTAINER | 0 |
+-------+---------+------+----------------------------------------------+-----------+-----------+
nr@clean-jammy-amd64:~$ systemctl stop systemd-resolved
nr@clean-jammy-amd64:~$ kill -9 $(pgrep dnsmasq)

Then, I start a new DNS server with a record which contains 10 CNAME redirects:

nr@clean-jammy-amd64:~$ sudo dnsmasq --cname=test10.lan,test9.lan --cname=test9.lan,test8.lan --cname=test8.lan,test7.lan --cname=test7.lan,test6.lan --cname=test6.lan,test5.lan --cname=test5.lan,test4.lan --cname=test4.lan,test3.lan --cname=test3.lan,test2.lan --cname=test2.lan,test1.lan --cname=test1.lan,test0.lan -k -i lxdbr0 -z -I lo --host-record=test0.lan,192.168.122.143

...

In the container, I have systemd installed from focal-proposed, and I was able to successfully make the query despite more than 8 CNAME redirects:

root@focal:~# apt policy systemd
systemd:
  Installed: 245.4-4ubuntu3.23
  Candidate: 245.4-4ubuntu3.23
  Version table:
 *** 245.4-4ubuntu3.23 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     245.4-4ubuntu3.22 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     245.4-4ubuntu3.20 500
        500 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages
     245.4-4ubuntu3 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages
root@focal:~# resolvectl query test10.lan
test10.lan: 192.168.122.143 -- link: eth0
            (test0.lan)

-- Information acquired via protocol DNS in 70.3ms.
-- Data is authenticated: no

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 245.4-4ubuntu3.23

---------------
systemd (245.4-4ubuntu3.23) focal; urgency=medium

  [ Nick Rosbrook ]
  * core/device: ignore DEVICE_FOUND_UDEV bit on switching root (LP: #2037281)
    File: debian/patches/lp2037281-core-device-ignore-DEVICE_FOUND_UDEV-bit-on-switching-roo.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=7793563bb38a84a3dc6bc0da1c08546c3b915ab8
  * dns-query: bump CNAME_MAX to 16 (LP: #2024009)
    File: debian/patches/lp2024009-dns-query-bump-CNAME_MAX-to-16.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=193899d103d44c642d362e9916b14df844ec702f
  * Fall back to kexec when no kexec binary exists (LP: #1969365)
    File: debian/patches/lp1969365-Fall-back-to-kexec-when-no-kexec-binary-exists.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3934f3794427dee4e72824998dd4c6e6d5875289
  * test: ignore LXC filesystem when checking for writable locations (LP: #2029352)
    File: debian/patches/lp2029352-test-ignore-LXC-filesystem-when-checking-for-writable-loc.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=70facbfbf54c4ffb31ba392dbe3fec3084fdf3bc

  [ Heitor Alves de Siqueira ]
  * core/mount: adjust deserialized state based on /proc/self/mountinfo (LP: #1837227)
    Author: Heitor Alves de Siqueira
    File: debian/patches/lp1837227-core-mount-adjust-deserialized-state-based-on-proc-self-m.patch
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=a0a749953d309f48bc45140102adf205d1071c4d

 -- Nick Rosbrook <email address hidden> Tue, 21 Nov 2023 16:10:21 -0500

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.