systemd: /etc/dhcp/dhclient-enter-hooks.d/resolved error

[Mason Loring Bliss]

[impact]

with systemd-resolved disabled, dhclient doesn't correctly notify resolvconf about dns server(s)

[test case]

install resolvconf and ifupdown and disable systemd-resolved and systemd-networkd, use ifupdown to get a dhcp address where the lease includes a dns nameserver, verify resolvconf is using that dhcp-provided nameserver

[regression potential]

failure to correctly notify systemd-resolved about new dhclient-provided nameserver(s)

[scope]

this is needed for f and earlier

in g and later the hook script is moved to the isc-dhcp package, and edited to correctly check is-enabled systemd-resolved instead of only checking for the existence of the binary

[original description]

The functionality exists to allow users to revert to the traditional ifupdown
package for network configuration. Alongside this, systemd's often-buggy
resolver can be disabled. However, there's a logic error in the systemd-
supplied /etc/dhcp/dhclient-enter-hooks.d/resolved that prevents the system
from populating /etc/resolv.conf properly when systemd-resolved is disabled.
The issue is here:

    if [ -x /lib/systemd/systemd-resolved ] ; then

Instead of checking to see if the systemd-resolved service is enabled or
active, which would be the correct behaviour, this checks for the existence of
a binary, assuming that if it exists it's supposed to be used.

I've not tested this in the absence of resolvconf, but if systemd-resolved
isn't enabled, it's difficult to imagine this code wanting to run. I've tested
this with resolvconf and ifupdown driving dhclient, and it corrects the
behaviour that was broken with the introduction of systemd-resolved.

I'm attaching a patch, and am also including it here for easy access:

*** resolved.broken 2019-11-19 15:01:28.785588838 +0000
--- resolved 2019-11-19 15:08:06.519430073 +0000
***************
*** 14,20 ****
  # (D) = master script downs interface
  # (-) = master script does nothing with this

! if [ -x /lib/systemd/systemd-resolved ] ; then
          # For safety, first undefine the nasty default make_resolv_conf()
          make_resolv_conf() { : ; }
          case "$reason" in
--- 14,21 ----
  # (D) = master script downs interface
  # (-) = master script does nothing with this

! systemctl is-active systemd-resolved > /dev/null 2>&1
! if [ $? -eq 0 ]; then
          # For safety, first undefine the nasty default make_resolv_conf()
          make_resolv_conf() { : ; }
          case "$reason" in

[Mason Loring Bliss (y-mason)]

sdezial notes this being terser. I do the long form out of superstitious awe
at the notion of a return code of zero being "true", even though it always
is, but this would be terser and also correct:

    if systemctl is-active systemd-resolved > /dev/null 2>&1; then

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "Check for active service, not existence of binary." seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Dan Streetman (ddstreet)]

I think this is a dup of bug 1745463, which might not be the answer you were hoping for.

[Mason Loring Bliss (y-mason)]

Interesting. But the code is still incorrect, with my patch correcting it,
so I guess we'll see how it goes. I can either fix it locally or go elsewhere,
but I'm hoping it's simply fixed in the distribution.

Thanks for the pointer.

[Seth Arnold (seth-arnold)]

I thought we were going to some effort to reduce the number of systems where resolveconf was going to be used, or even remove it from the distro entirely.

Thanks

[Steve Langasek (vorlon)]

For Ubuntu 20.04, resolvconf will not be allowed to manage /etc/resolv.conf, this will always be managed via systemd-resolved. So while you're right that this is a bug in the hook, it will not be a priority to fix since the code will be rewritten and replaced.

> Alongside this, systemd's often-buggy resolver can be disabled.

The vast majority of Ubuntu users are using systemd-resolved without incident or complaint. If you know of specific problems with it, we can best help you (and all other Ubuntu users) if you give us bug reports for those problems.

[Dan Streetman (ddstreet)]

bug 1890609 removed the hook script from systemd (by moving it into the isc-dhcp package)

[Thayne (thayne-u)]

> If you know of specific problems with it, we can best help you (and all other Ubuntu users) if you give us bug reports for those problems.

There's this: https://github.com/systemd/systemd/issues/5755

Also, systemd-resolved does not have an equivalent of dnsmasq's "all-servers" option, which I need. And because of this bug, I can't run dnsmasq instead of systemd-resolved. I have been able to get something working with both systemd-resolved and dnsmasq running, but having two local dns resolvers is unnecessarily redundant, and more complicated than if I could just run dnsmasq.

See my comments on https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1745463

[GeekSmith (lixo-geeksmith)]

@thayne-u

They don't care. There is ample reason to implement a fix for this, but they refuse to do what's best for users because they don't want us to run any software that doesn't start with "systemd". This is a very simple bug with a very simple fix, but they refuse to fix their defects.

I've used Ubuntu as my primary OS since Warty Warthog. I've installed and supported it on countless desktop and server systems. Something changed in the culture a few years back -- they stopped listening to their users, around the time that Jono Bacon left. Now the community has no advocate to influence development. Sadly, that shift in attitude is reflected with this bug and the one like it that I filed in 2018.

[Dan Streetman (ddstreet)]

@lixo-geeksmith I have the patch for this queued up for the next systemd upload, sorry for the long delay.

[GeekSmith (lixo-geeksmith)]

@ddstreet

Thank you! It has been a long time coming, so I'm glad to see this fixed.

[Daniel Richard G. (skunk)]

Thank you @ddstreet, I'm happy to see this as well. I'd like to get rid of the workaround I've been using for this issue:

# dpkg-divert --divert /etc/dhcp/dhclient-enter-hooks.d/resolved.DISABLED --rename /etc/dhcp/dhclient-enter-hooks.d/resolved

[Dan Streetman (ddstreet)]

My staging ppa has the 'next' version that i'll be uploading soon (possibly with other patches), if you want to test it to verify it does fix this for you:

https://launchpad.net/~ddstreet/+archive/ubuntu/systemd

[Brian Murray (brian-murray)]

Hello Mason, or anyone else affected,

Accepted systemd into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/245.4-4ubuntu3.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (245.4-4ubuntu3.8) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

asterisk/1:16.2.1~dfsg-2ubuntu1 (armhf)
gvfs/1.44.1-1ubuntu1 (amd64)
linux-oem-5.6/5.6.0-1057.61 (amd64)
munin/2.0.56-1ubuntu1 (ppc64el)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Mason Loring Bliss (y-mason)]

Hey there. Thanks for your time on this. I'll try to supply positive confirmation over the weekend.

[Mason Loring Bliss (y-mason)]

I've tested this, and under the noted conditions resolvconf no longer has an issue updating /etc/resolv.conf. Thank you for your time and attention to detail!

Unless someone else weighs in noting a problem, from my perspective it seems like this bug can be closed now, as you've corrected the issue. Thank you.

[Daniel Richard G. (skunk)]

FWIW, the fix in focal-proposed looks good on my end as well.

I can confirm that the /etc/dhcp/dhclient-enter-hooks.d/resolved script now has the is-enabled check, and while I won't be able to test out resolvconf, I regard the updated conditional as equivalent to my previous known-good workaround (renaming the script to resolved.DISABLED, so it is not sourced by dhclient-script).

Greatly appreciate the bug squash!

[Dan Streetman (ddstreet)]

marking as verified for focal based on above feedback

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 245.4-4ubuntu3.10

---------------
systemd (245.4-4ubuntu3.10) focal-security; urgency=medium

  * SECURITY UPDATE: DoS via DHCP FORCERENEW
    - debian/patches/CVE-2020-13529.patch: tentatively ignore FORCERENEW
      command in src/libsystemd-network/sd-dhcp-client.c.
    - CVE-2020-13529
  * SECURITY UPDATE: denial of service via stack exhaustion
    - debian/patches/CVE-2021-33910.patch: do not use strdupa() on a path
      in src/basic/unit-name.c.
    - CVE-2021-33910

 -- Marc Deslauriers <email address hidden> Tue, 20 Jul 2021 07:39:51 -0400

[Chris Halse Rogers (raof)]

Hello Mason, or anyone else affected,

Accepted systemd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.52 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted systemd (237-3ubuntu10.52) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

casync/2+61.20180112-1 (s390x)
nftables/0.8.2-1 (amd64)
lxc/3.0.3-0ubuntu1~18.04.1 (amd64, i386, s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Dan Streetman (ddstreet)]

as systemd was respun due to bug 1942899, and only the one udev (hwdb) patch was reverted which shouldn't affect this at all, I'm remarking this as verified still based on testing for the previous version above.

[Dan Streetman (ddstreet)]

heh, please ignore the above comment, the current focal systemd was respun, but the fix for this was already released for focal, this only needs bionic verified now :)

[Dan Streetman (ddstreet)]

root@lp1853164-b:~# dpkg -l systemd|grep systemd
ii systemd 237-3ubuntu10.51 amd64 system and service manager
root@lp1853164-b:~# dpkg -l | grep -E 'ifupdown|resolvconf'
ii ifupdown 0.8.17ubuntu1.1 amd64 high level tools to configure network interfaces
ii resolvconf 1.79ubuntu10.18.04.3 all name server information handler
root@lp1853164-b:~# grep eth0 /etc/network/interfaces
auto eth0
iface eth0 inet dhcp
root@lp1853164-b:~# ip a show dev eth0
222: eth0@if223: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:16:1c:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.202.51.62/24 brd 10.202.51.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe16:1c78/64 scope link
       valid_lft forever preferred_lft forever
root@lp1853164-b:~# systemctl is-enabled systemd-resolved
disabled
root@lp1853164-b:~# systemctl is-enabled systemd-networkd
disabled
root@lp1853164-b:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

root@lp1853164-b:~#

root@lp1853164-b:~# dpkg -l systemd|grep systemd
ii systemd 237-3ubuntu10.52 amd64 system and service manager
root@lp1853164-b:~# dpkg -l | grep -E 'ifupdown|resolvconf'
ii ifupdown 0.8.17ubuntu1.1 amd64 high level tools to configure network interfaces
ii resolvconf 1.79ubuntu10.18.04.3 all name server information handler
root@lp1853164-b:~# grep eth0 /etc/network/interfaces
auto eth0
iface eth0 inet dhcp
root@lp1853164-b:~# ip a show dev eth0
224: eth0@if225: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:16:1c:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.202.51.62/24 brd 10.202.51.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe16:1c78/64 scope link
       valid_lft forever preferred_lft forever
root@lp1853164-b:~# systemctl is-enabled systemd-resolved
disabled
root@lp1853164-b:~# systemctl is-enabled systemd-networkd
disabled
root@lp1853164-b:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

nameserver 10.202.51.1
search lxd
root@lp1853164-b:~#

[Chris Halse Rogers (raof)]

The verification of the Stable Release Update for systemd has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package systemd - 237-3ubuntu10.52

---------------
systemd (237-3ubuntu10.52) bionic; urgency=medium

  * d/extra/dhclient-enter-resolved-hook:
    Reset start limit counter for systemd-resolved in dhclient hook
    (LP: #1939255)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=ea6710476dde78e8595274c3c4ba7acca6d5162c
  * d/p/lp1934147/0001-core-add-a-new-unit-method-catchup.patch,
    d/p/lp1934147/0002-cgroup-do-catchup-for-unit-cgroup-inotify-watch-file.patch,
    d/p/lp1934147/0003-core-Make-sure-cgroup_oom_queue-is-flushed-on-manage.patch:
    Catch up on cgroup empty inotify after reexec/reload (LP: #1934147)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=5ef61bd930612a90ce3ed9105cbadc5ff97b6ffc
  * d/p/lp1934981-correct-suspend-then-sleep-string.patch:
    Fix sleep verb used by logind during suspend-then-hibernate
    (LP: #1934981)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=1ade873a41ad018a5e07f10775738c6eb8c82310
  * d/extra/dhclient-enter-resolved-hook:
    Check is-enabled systemd-resolved in dhclient hook (LP: #1853164)
    https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=774c2f82a39a88fa0fd8b2adbfa0b8a8c3cd1fb5

 -- Dan Streetman <email address hidden> Thu, 26 Aug 2021 10:20:40 -0400