Failure to start after upgrade if inline-signing is missing

[Andreas Hasenack]

[Impact]

If a user upgrades bind9 to the current version in Kinetic and Jammy, 9.18.12, and at least one of their zone definitions uses dnssec-policy without a configured 'allow-update' or 'update-policy' with inline-signing not marked as yes, then their DNS will fail to start back up. This is due to a change in 9.18.7 where adding 'inline-signing yes;' to the zone definition in this situation is necessary.

This fix should be added to avoid the regression when upgrading.

The issue is fixed by adding a patch that reverts non-test portions of the following commit from upstream:

https://gitlab.isc.org/isc-projects/bind9/-/commit/7be0594be923802a06a0dc5459e1869d99154f1b?page=2

It reverts the check library to only enforce inline-signing not being off, and defaults inline-signing back to yes in this specific situation.

[Test Plan]

# lxc launch images:ubuntu/jammy test-bind9
# lxc exec test-bind9 bash
# apt update && apt dist-upgrade -y

# apt install bind9 -y

# cat <<EOF >/etc/bind/named.conf.options
options { directory "/var/cache/bind"; listen-on port 53 { 127.0.0.1; }; allow-query { any; }; recursion yes; };
EOF

# cat <<EOF >/etc/bind/named.conf.local
zone "localdomain.test" { type master; file "/etc/bind/zones/forward.localdomain.test"; dnssec-policy default;};
EOF

# mkdir -p /etc/bind/zones/

# cat <<EOF >/etc/bind/zones/forward.localdomain.test
\$TTL 604800
@ IN SOA localdomain.test. root.localdomain.test. (
         2 ; Serial
    604800 ; Refresh
     86400 ; Retry
   2419200 ; Expire
    604800 ) ; Negative Cache TTL
;
@ IN NS ns.localdomain.test.
ns IN A 127.0.0.1
EOF

# systemctl restart bind9
> Before the fix:
Job for named.service failed because the control process exited with error code.
See "systemctl status named.service" and "journalctl -xeu named.service" for details.

> After the fix there is no output since the restart succeeds

[Where problems could occur]

If problems were to occur, they would likely show up in interactions with upstream changes from 9.18.7 to 9.18.12. Since upstream now assumes that inline-signing is enforced with dnssec-policy without a configured 'allow-update' or 'update-policy', they may have made changes that reflect this.

Likewise, this will reintroduce the possible regression that the change was meant to block, which is when allow-update or update-policy are added to the zone, the default active inline-signing will be removed. This would cause named to maintain, sign, and serve an unsigned zone file. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing

[Original Description]

bind9 releases 9.16.33 and 9.18.7 introduced a change[1] that can make bind9 fail to restart after an upgrade from earlier releases.

A bind9 server that was serving a DNSSEC zone before will refuse to start after this update if the administrator was relying on the "inline-signing yes" silent default previously.

For example, if you had a zone like this:

/var/lib/bind/db.example.fake:
$TTL 3600
@ IN SOA example.fake. root.example.fake. (
                              2 ; Serial
                         3600 ; Refresh
                          86400 ; Retry
                        2419200 ; Expire
                         3600 ) ; Negative Cache TTL
;
@ IN NS ns.example.fake.
@ IN A 192.168.10.1
ns IN A 192.168.10.1
banana IN A 192.168.10.2
avocado IN A 192.168.10.3

Configured like this (note the absence of "inline-signing yes"):
zone "example.fake" {
    type master;
    file "/var/lib/bind/db.example.fake";
    dnssec-policy default;
};

When you upgrade bind9 to the updated version in jammy and kinetic, the named service will fail to start (example below depicted from a jammy update):
Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration from '/etc/bind/named.conf'
Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: /etc/bind/named.conf.local:12: 'inline-signing yes;' must also be configured explicitly for zones using dnssec-policy without a configured 'allow-update' or 'update-policy'. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing
Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration: failure
Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: exiting (due to fatal error)
Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Control process exited, code=exited, status=1/FAILURE
Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Failed with result 'exit-code'.
Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: Failed to start BIND Domain Name Server.

The apt transaction won't fail, but will also not flag that the service is down:

Setting up bind9 (1:9.18.12-0ubuntu0.22.04.1) ...
named-resolvconf.service is a disabled or a static unit not running, not starting it.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142.
Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.1) ...
Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.1) ...
Processing triggers for ufw (0.36.1-4build1) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...

root@j-bind9-mre-upgrade:~# systemctl status bind9
× named.service - BIND Domain Name Server
     Loaded: loaded (/lib/systemd/system/named.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-04-10 18:31:18 UTC; 3s ago
       Docs: man:named(8)
    Process: 4346 ExecStart=/usr/sbin/named $OPTIONS (code=exited, status=1/FAILURE)
        CPU: 26ms

We haven't gotten reports about this yet, and maybe we won't, but it's a risky update.

I briefly checked the named-checkconf and named-checkzone utilities, to see if they would catch this configuration problem, but they just look at syntax errors, not this type of misconfiguration, so they don't flag it. If they did, maybe we could do something in preinst to fail the upgrade before the service is brought down.

The focal version of the bind9 MRE is currently in review, and will probably be halted until we have a better understanding of this issue and how to deal with it.

Ideas welcome.

1. https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing

[Andreas Hasenack (ahasenack)]

Upstream comment from a end-user who was bitten by this bug during an update, with a reply from usptream: https://gitlab.isc.org/isc-projects/bind9/-/issues/3381#note_317347

[Andreas Hasenack (ahasenack)]

Oh, and look at that: the bug report at https://gitlab.isc.org/isc-projects/bind9/-/issues/3381 came from an ubuntu user of 9.18.1-1ubuntu1, the version we shipped in Jammy. That's the bug report that prompted this backwards incompatible change in later versions.

[Lena Voytek (lvoytek)]

Marking kinetic as wont-fix since it's eol. jammy likely won't be fixed either since that will more likely just cause further regressions for users while those affected have likely already dealt with it, but I'll mark it as triaged for now in case we do decide to fix it later on