Activity log for bug #2015793

Date Who What changed Old value New value Message
2023-04-10 18:37:19 Andreas Hasenack bug added bug
2023-04-10 18:38:46 Andreas Hasenack nominated for series Ubuntu Kinetic
2023-04-10 18:38:46 Andreas Hasenack bug task added bind9 (Ubuntu Kinetic)
2023-04-10 18:38:46 Andreas Hasenack nominated for series Ubuntu Jammy
2023-04-10 18:38:46 Andreas Hasenack bug task added bind9 (Ubuntu Jammy)
2023-04-10 18:40:35 Andreas Hasenack description bind9 releases 9.16.33 and 9.18.7 introduced a change[1] that can make bind9 fail to restart after an upgrade from earlier releases. A bind9 server that was serving a DNSSEC zone before will refuse to start after this update if the administrator was relying on the "inline-signing yes" silent default previously. For example, if you had a zone like this: /var/lib/bind/db.example.fake: $TTL 3600 @ IN SOA example.fake. root.example.fake. ( 2 ; Serial 3600 ; Refresh 86400 ; Retry 2419200 ; Expire 3600 ) ; Negative Cache TTL ; @ IN NS ns.example.fake. @ IN A 192.168.10.1 ns IN A 192.168.10.1 banana IN A 192.168.10.2 avocado IN A 192.168.10.3 Configured like this (note the absence of "inline-signing yes"): zone "example.fake" { type master; file "/var/lib/bind/db.example.fake"; dnssec-policy default; }; When you upgrade bind9 to the updated version in focal and kinetic, the named service will fail to start (example below depicted from a jammy update): Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration from '/etc/bind/named.conf' Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: /etc/bind/named.conf.local:12: 'inline-signing yes;' must also be configured explicitly for zones using dnssec-policy without a configured 'allow-update' or 'update-policy'. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration: failure Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: exiting (due to fatal error) Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Control process exited, code=exited, status=1/FAILURE Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Failed with result 'exit-code'. Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: Failed to start BIND Domain Name Server. The apt transaction won't fail, but will also not flag that the service is down: Setting up bind9 (1:9.18.12-0ubuntu0.22.04.1) ... named-resolvconf.service is a disabled or a static unit not running, not starting it. Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.1) ... Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.1) ... Processing triggers for ufw (0.36.1-4build1) ... Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... root@j-bind9-mre-upgrade:~# systemctl status bind9 × named.service - BIND Domain Name Server Loaded: loaded (/lib/systemd/system/named.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Mon 2023-04-10 18:31:18 UTC; 3s ago Docs: man:named(8) Process: 4346 ExecStart=/usr/sbin/named $OPTIONS (code=exited, status=1/FAILURE) CPU: 26ms We haven't gotten reports about this yet, and maybe we won't, but it's a risky update. I briefly checked the named-checkconf and named-checkzone utilities, to see if they would catch this configuration problem, but they just look at syntax errors, not this type of misconfiguration, so they don't flag it. If they did, maybe we could do something in preinst to fail the upgrade before the service is brought down. The focal version of the bind9 MRE is currently in review, and will probably be halted until we have a better understanding of this issue and how to deal with it. Ideas welcome. 1. https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing bind9 releases 9.16.33 and 9.18.7 introduced a change[1] that can make bind9 fail to restart after an upgrade from earlier releases. A bind9 server that was serving a DNSSEC zone before will refuse to start after this update if the administrator was relying on the "inline-signing yes" silent default previously. For example, if you had a zone like this: /var/lib/bind/db.example.fake: $TTL 3600 @ IN SOA example.fake. root.example.fake. (                               2 ; Serial                          3600 ; Refresh                           86400 ; Retry                         2419200 ; Expire                          3600 ) ; Negative Cache TTL ; @ IN NS ns.example.fake. @ IN A 192.168.10.1 ns IN A 192.168.10.1 banana IN A 192.168.10.2 avocado IN A 192.168.10.3 Configured like this (note the absence of "inline-signing yes"): zone "example.fake" {     type master;     file "/var/lib/bind/db.example.fake";     dnssec-policy default; }; When you upgrade bind9 to the updated version in jammy and kinetic, the named service will fail to start (example below depicted from a jammy update): Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration from '/etc/bind/named.conf' Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: /etc/bind/named.conf.local:12: 'inline-signing yes;' must also be configured explicitly for zones using dnssec-policy without a configured 'allow-update' or 'update-policy'. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration: failure Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: exiting (due to fatal error) Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Control process exited, code=exited, status=1/FAILURE Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Failed with result 'exit-code'. Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: Failed to start BIND Domain Name Server. The apt transaction won't fail, but will also not flag that the service is down: Setting up bind9 (1:9.18.12-0ubuntu0.22.04.1) ... named-resolvconf.service is a disabled or a static unit not running, not starting it. Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.1) ... Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.1) ... Processing triggers for ufw (0.36.1-4build1) ... Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... root@j-bind9-mre-upgrade:~# systemctl status bind9 × named.service - BIND Domain Name Server      Loaded: loaded (/lib/systemd/system/named.service; enabled; vendor preset: enabled)      Active: failed (Result: exit-code) since Mon 2023-04-10 18:31:18 UTC; 3s ago        Docs: man:named(8)     Process: 4346 ExecStart=/usr/sbin/named $OPTIONS (code=exited, status=1/FAILURE)         CPU: 26ms We haven't gotten reports about this yet, and maybe we won't, but it's a risky update. I briefly checked the named-checkconf and named-checkzone utilities, to see if they would catch this configuration problem, but they just look at syntax errors, not this type of misconfiguration, so they don't flag it. If they did, maybe we could do something in preinst to fail the upgrade before the service is brought down. The focal version of the bind9 MRE is currently in review, and will probably be halted until we have a better understanding of this issue and how to deal with it. Ideas welcome. 1. https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing
2023-04-11 19:09:20 Simon Déziel bug added subscriber Simon Déziel
2023-04-11 21:22:58 Launchpad Janitor merge proposal linked https://code.launchpad.net/~lvoytek/ubuntu/+source/bind9/+git/bind9/+merge/440776
2023-04-11 21:38:13 Launchpad Janitor merge proposal linked https://code.launchpad.net/~lvoytek/ubuntu/+source/bind9/+git/bind9/+merge/440777
2023-04-11 22:18:14 Lena Voytek bind9 (Ubuntu Jammy): assignee Lena Voytek (lvoytek)
2023-04-11 22:18:16 Lena Voytek bind9 (Ubuntu Kinetic): assignee Lena Voytek (lvoytek)
2023-04-11 22:18:19 Lena Voytek bind9 (Ubuntu Jammy): status New In Progress
2023-04-11 22:18:21 Lena Voytek bind9 (Ubuntu Kinetic): status New In Progress
2023-04-11 22:18:56 Lena Voytek description bind9 releases 9.16.33 and 9.18.7 introduced a change[1] that can make bind9 fail to restart after an upgrade from earlier releases. A bind9 server that was serving a DNSSEC zone before will refuse to start after this update if the administrator was relying on the "inline-signing yes" silent default previously. For example, if you had a zone like this: /var/lib/bind/db.example.fake: $TTL 3600 @ IN SOA example.fake. root.example.fake. (                               2 ; Serial                          3600 ; Refresh                           86400 ; Retry                         2419200 ; Expire                          3600 ) ; Negative Cache TTL ; @ IN NS ns.example.fake. @ IN A 192.168.10.1 ns IN A 192.168.10.1 banana IN A 192.168.10.2 avocado IN A 192.168.10.3 Configured like this (note the absence of "inline-signing yes"): zone "example.fake" {     type master;     file "/var/lib/bind/db.example.fake";     dnssec-policy default; }; When you upgrade bind9 to the updated version in jammy and kinetic, the named service will fail to start (example below depicted from a jammy update): Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration from '/etc/bind/named.conf' Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: /etc/bind/named.conf.local:12: 'inline-signing yes;' must also be configured explicitly for zones using dnssec-policy without a configured 'allow-update' or 'update-policy'. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration: failure Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: exiting (due to fatal error) Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Control process exited, code=exited, status=1/FAILURE Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Failed with result 'exit-code'. Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: Failed to start BIND Domain Name Server. The apt transaction won't fail, but will also not flag that the service is down: Setting up bind9 (1:9.18.12-0ubuntu0.22.04.1) ... named-resolvconf.service is a disabled or a static unit not running, not starting it. Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.1) ... Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.1) ... Processing triggers for ufw (0.36.1-4build1) ... Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... root@j-bind9-mre-upgrade:~# systemctl status bind9 × named.service - BIND Domain Name Server      Loaded: loaded (/lib/systemd/system/named.service; enabled; vendor preset: enabled)      Active: failed (Result: exit-code) since Mon 2023-04-10 18:31:18 UTC; 3s ago        Docs: man:named(8)     Process: 4346 ExecStart=/usr/sbin/named $OPTIONS (code=exited, status=1/FAILURE)         CPU: 26ms We haven't gotten reports about this yet, and maybe we won't, but it's a risky update. I briefly checked the named-checkconf and named-checkzone utilities, to see if they would catch this configuration problem, but they just look at syntax errors, not this type of misconfiguration, so they don't flag it. If they did, maybe we could do something in preinst to fail the upgrade before the service is brought down. The focal version of the bind9 MRE is currently in review, and will probably be halted until we have a better understanding of this issue and how to deal with it. Ideas welcome. 1. https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing [Impact] If a user upgrades bind9 to the current version in Kinetic and Jammy, 9.18.12, and at least one of their zone definitions uses dnssec-policy without a configured 'allow-update' or 'update-policy' with inline-signing not marked as yes, then their DNS will fail to start back up. This is due to a change in 9.18.7 where adding 'inline-signing yes;' to the zone definition in this situation is necessary. This fix should be added to avoid the regression when upgrading. The issue is fixed by adding a patch that reverts non-test portions of the following commit from upstream: https://gitlab.isc.org/isc-projects/bind9/-/commit/7be0594be923802a06a0dc5459e1869d99154f1b?page=2 It reverts the check library to only enforce inline-signing not being off, and defaults inline-signing back to yes in this specific situation. [Test Plan] # lxc launch images:ubuntu/jammy test-bind9 # lxc exec test-bind9 bash # apt update && apt dist-upgrade -y # apt install bind9 -y # cat <<EOF >/etc/bind/named.conf.options options { directory "/var/cache/bind"; listen-on port 53 { 127.0.0.1; }; allow-query { any; }; recursion yes; }; EOF # cat <<EOF >/etc/bind/named.conf.local zone "localdomain.test" { type master; file "/etc/bind/zones/forward.localdomain.test"; dnssec-policy default;}; EOF # mkdir -p /etc/bind/zones/ # cat <<EOF >/etc/bind/zones/forward.localdomain.test \$TTL 604800 @ IN SOA localdomain.test. root.localdomain.test. ( 2 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL ; @ IN NS ns.localdomain.test. ns IN A 127.0.0.1 EOF # systemctl restart bind9 > Before the fix: Job for named.service failed because the control process exited with error code. See "systemctl status named.service" and "journalctl -xeu named.service" for details. > After the fix there is no output since the restart succeeds [Where problems could occur] If problems were to occur, they would likely show up in interactions with upstream changes from 9.18.7 to 9.18.12. Since upstream now assumes that inline-signing is enforced with dnssec-policy without a configured 'allow-update' or 'update-policy', they may have made changes that reflect this. Likewise, this will reintroduce the possible regression that the change was meant to block, which is when allow-update or update-policy are added to the zone, the default active inline-signing will be removed. This would cause named to maintain, sign, and serve an unsigned zone file. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing [Original Description] bind9 releases 9.16.33 and 9.18.7 introduced a change[1] that can make bind9 fail to restart after an upgrade from earlier releases. A bind9 server that was serving a DNSSEC zone before will refuse to start after this update if the administrator was relying on the "inline-signing yes" silent default previously. For example, if you had a zone like this: /var/lib/bind/db.example.fake: $TTL 3600 @ IN SOA example.fake. root.example.fake. ( 2 ; Serial 3600 ; Refresh 86400 ; Retry 2419200 ; Expire 3600 ) ; Negative Cache TTL ; @ IN NS ns.example.fake. @ IN A 192.168.10.1 ns IN A 192.168.10.1 banana IN A 192.168.10.2 avocado IN A 192.168.10.3 Configured like this (note the absence of "inline-signing yes"): zone "example.fake" { type master; file "/var/lib/bind/db.example.fake"; dnssec-policy default; }; When you upgrade bind9 to the updated version in jammy and kinetic, the named service will fail to start (example below depicted from a jammy update): Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration from '/etc/bind/named.conf' Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: /etc/bind/named.conf.local:12: 'inline-signing yes;' must also be configured explicitly for zones using dnssec-policy without a configured 'allow-update' or 'update-policy'. See https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: loading configuration: failure Apr 10 18:31:18 j-bind9-mre-upgrade named[4347]: exiting (due to fatal error) Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Control process exited, code=exited, status=1/FAILURE Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: named.service: Failed with result 'exit-code'. Apr 10 18:31:18 j-bind9-mre-upgrade systemd[1]: Failed to start BIND Domain Name Server. The apt transaction won't fail, but will also not flag that the service is down: Setting up bind9 (1:9.18.12-0ubuntu0.22.04.1) ... named-resolvconf.service is a disabled or a static unit not running, not starting it. Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.1) ... Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.1) ... Processing triggers for ufw (0.36.1-4build1) ... Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... root@j-bind9-mre-upgrade:~# systemctl status bind9 × named.service - BIND Domain Name Server Loaded: loaded (/lib/systemd/system/named.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Mon 2023-04-10 18:31:18 UTC; 3s ago Docs: man:named(8) Process: 4346 ExecStart=/usr/sbin/named $OPTIONS (code=exited, status=1/FAILURE) CPU: 26ms We haven't gotten reports about this yet, and maybe we won't, but it's a risky update. I briefly checked the named-checkconf and named-checkzone utilities, to see if they would catch this configuration problem, but they just look at syntax errors, not this type of misconfiguration, so they don't flag it. If they did, maybe we could do something in preinst to fail the upgrade before the service is brought down. The focal version of the bind9 MRE is currently in review, and will probably be halted until we have a better understanding of this issue and how to deal with it. Ideas welcome. 1. https://kb.isc.org/docs/dnssec-policy-requires-dynamic-dns-or-inline-signing
2023-04-12 14:10:57 Andreas Hasenack bug watch added https://gitlab.isc.org/isc-projects/bind9/-/issues/3381
2023-07-25 15:18:43 Lena Voytek bind9 (Ubuntu Jammy): status In Progress Triaged
2023-07-25 15:18:45 Lena Voytek bind9 (Ubuntu Kinetic): status In Progress Won't Fix
2023-07-25 15:18:54 Lena Voytek bind9 (Ubuntu): status New Won't Fix