MAAS randomly doesn't generate zone files on restart

[Radu Popescu]

MAAS installed by snap. Happens the same with both 3.4.0 and 3.4.1 (rev 34087).
If it fails to generate the zone files (including reverse DNS), here's all from /var/snap/maas/current/bind/:

bind.keys
cache
db.0
db.127
db.255
db.empty
db.local
keys.conf.maas
named.conf
named.conf.default-zones
named.conf.local
named.conf.maas
named.conf.options
named.conf.options.inside.maas
named.conf.rndc.maas
rndc.conf.maas
zones.rfc1918

Obviously, named service complains about files not being found:

18-Mar-2024 21:37:51.189 zone 116.31.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.116.31.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 7.45.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.7.45.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 9.74.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.9.74.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 2.45.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.2.45.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 6.45.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.6.45.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 15.1.16.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.15.1.16.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 3.45.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.3.45.10.in-addr.arpa failed: file not found
18-Mar-2024 21:37:51.189 zone 5.45.10.in-addr.arpa/IN: loading from master file /var/snap/maas/34087/bind/zone.5.45.10.in-addr.arpa failed: file not found

Also happens to "maas" zone and any other custom zones defined.

Steps to reproduce:
- scenario 1 - snap refreshed automatically and restarted.
- scenario 2 - MAAS goes through a backup process as described here: https://maas.io/docs/backing-up-maas, like:
snap stop maas
snap save maas
snap export-snapshot
snap forget
snap restart maas

Temporary workarounds:
- scenario 1 - stop automatic refresh (snap refresh --hold=forever)
- scenario 2 - after last "snap restart maas", verify custom zone file present and if it's missing, restart maas again.

Problem usually fixed by "snap restart maas".

[Christian Grabowski (cgrabowski)]

Hi there, can you please provide a sos-report from when this occurs?

[Radu Popescu (rpopescu-cbs)]

Sos report attached.

[Radu Popescu (rpopescu-cbs)]

MAAS is all-in-one snap install. Backup is done using "Clean Snap backup" instructions from here: https://maas.io/docs/how-to-back-vgup-maas. Logs should be around 1AM every Sunday. To be honest, not sure how far back sosreport covers.

[Bill Wear (billwear)]

Given that that zone files can't be found, I'd think that the snap's handling of persistent storage might be failing, or that zone files are not being correctly (re)generated. Can't reproduce, but can't see how this isn't (most likely) an actual bug. Moving to triaged.