Charmed Designate is continuously updating the zones even when there is no activity on the cloud.
While this is not a big issue when having one zone. However because of the frequency and potentially having multiple zones updated at the same time, this can create a resource shortage on controller nodes hosting designate units.
This does not happen on a Devstack deployment test.
Issue is seen at least on :
* Juju 2.9.38
* Openstack : Focal-Yoga / Jammy-Yoga / Jammy-Zed
* Designate charm : yoga/stable & latest/edge
* Designate-bind charm : yoga/stable & latest/edge
* in non HA using based on the bundle for testing here : https://opendev.org/openstack/charm-designate/src/branch/master/src/tests/bundles/jammy-zed.yaml
* there are no resources in the cloud aside from 2~3 zones with 2 records in each zone.
#
# Updates from Designate-worker process on designate/0 unit
# 117 AXFR updates in 20 minutes for one zone from one designate unit
#
2023-01-20 03:48:13.773 467 DEBUG designate.dnsutils [None designate-manage - - - all - -] AXFR Successful for example0.com. do_axfr /usr/lib/python3/dist-packages/designate/dnsutils.py:370
2023-01-20 03:48:13.936 467 DEBUG designate.dnsutils [None designate-manage - - - all - -] AXFR Successful for example0.com. do_axfr /usr/lib/python3/dist-packages/designate/dnsutils.py:370
...
...
2023-01-20 04:08:18.319 467 DEBUG designate.dnsutils [None designate-manage - - - all - -] AXFR Successful for example0.com. do_axfr /usr/lib/python3/dist-packages/designate/dnsutils.py:370
ubuntu@juju-e4c10f-2-lxd-0:~$ sudo cat /var/log/designate/designate-worker.log | grep "AXFR Successful" | grep example0.com. | grep 2023-01-20 | wc -l
117
ubuntu@juju-e4c10f-2-lxd-0:~$ date
Fri Jan 20 04:09:20 UTC 2023
#
# Charmed Openstack Jammy Zed deployment
# Zones get updated constantly and the Serial keeps changing with time
#
ubuntu@maas:~$ date; openstack zone list
Fri Jan 20 03:46:40 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example0.com. | 1674186223 | ACTIVE |
| example1.com. | 1674186223 | ACTIVE |
+---------------+------------+--------+
ubuntu@maas:~$ sleep 300; date; openstack zone list
Fri Jan 20 03:51:48 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example0.com. | 1674186493 | ACTIVE |
| example1.com. | 1674186493 | ACTIVE |
+---------------+------------+--------+
ubuntu@maas:~$ sleep 300; date; openstack zone list
Fri Jan 20 04:06:01 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example0.com. | 1674187431 | ACTIVE |
| example1.com. | 1674187431 | ACTIVE |
+---------------+------------+--------+
#
# Devstack zone serial updates
# The zones doesn't get updated at all once all records are created, and the serial stays the same
#
ubuntu@devstack:~$ date; openstack zone list
Fri Jan 20 03:44:04 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example2.com. | 1674186154 | ACTIVE |
| example1.com. | 1674186154 | ACTIVE |
| example0.com. | 1674186154 | ACTIVE |
+---------------+------------+--------+
ubuntu@devstack:~$ sleep 300; date; openstack zone list
Fri Jan 20 03:49:11 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example2.com. | 1674186154 | ACTIVE |
| example1.com. | 1674186154 | ACTIVE |
| example0.com. | 1674186154 | ACTIVE |
+---------------+------------+--------+
ubuntu@devstack:~$ sleep 300; date; openstack zone list
Fri Jan 20 04:05:59 UTC 2023
+---------------+------------+--------+
| name | serial | status |
+---------------+------------+--------+
| example2.com. | 1674186154 | ACTIVE |
| example1.com. | 1674186154 | ACTIVE |
| example0.com. | 1674186154 | ACTIVE |
+---------------+------------+--------+
#
# From Devstack designate-worker logs
#
Once all zones and records are created no actions are seeing at all on the logs from designate-worker
--> Can be seen from devstack_designate-worker.log in the attached archive
--> /etc/designate/designate.conf is included as devstack_designate.conf
--> /etc/designate/pools.yaml is included as devstack_pools.yaml
#
# From Charmed Designate-worker logs
#
The Zones get checked many many times and are constantly getting updated
--> Can be seen from charm_designate-worker.log in the attached archive
--> /etc/designate/designate.conf is included as charm_designate.conf
--> /etc/designate/pools.yaml is included as charm_pools.yaml
#
# Devstack configuration
#
[[local|localrc]]
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
enable_service rabbit mysql key
enable_plugin openstack/designate https://opendev.org/openstack/designate
enable_service designate,designate-central,designate-api,designate-worker,designate-zone-manager,designate-producer,designate-mdns
DESIGNATE_BACKEND_DRIVER=bind9
I've finally found the root cause, and the source is not from Designate configuration but from the charm itself.
After shutting down the juju agent on the Designate unit, the updates suddenly stopped.
Looking more deeply, it seems that the update-status hook runs many handlers that has an impact and creates a modification on the deployment. In particular with the "configure_
Reactive handlers seen inside update-status (see unit-designate-
2023-01-30 04:46:57 INFO unit.designate/
2023-01-30 04:46:57 INFO unit.designate/
2023-01-30 04:46:57 INFO unit.designate/
2023-01-30 04:47:14 INFO unit.designate/
2023-01-30 04:47:14 INFO unit.designate/
2023-01-30 04:47:27 INFO unit.designate/
2023-01-30 04:47:27 INFO unit.designate/
2023-01-30 04:47:27 INFO unit.designate/
2023-01-30 04:47:27 INFO unit.designate/
2023-01-30 04:47:27 INFO unit.designate/
Seen in HA deployment :
2023-01-30 08:23:55 INFO unit.designate/
2023-01-30 08:23:55 INFO unit.designate/
I've made a quick patch setting the following "@reactive.