designate-bind-k8s non leader units are in error state

[Hemanth Nakkina]

Deployed sunbeam 2023.2/edge and enabled dns using command
`sunbeam enable dns --nameservers=testing.github.`

bind is in error state

$ juju status -m openstack bind
Model Controller Cloud/Region Version SLA Timestamp
openstack sbc01-controller sunbeam-k8s/localhost 3.4.2 unsupported 11:27:08Z

SAAS Status Store URL
grafana-dashboards active local admin/observability.grafana-dashboards
loki-logging active local admin/observability.loki-logging
microceph active local admin/openstack-machines.microceph
prometheus-receive-remote-write active local admin/observability.prometheus-receive-remote-write

App Version Status Scale Charm Channel Rev Address Exposed Message
bind waiting 3 designate-bind-k8s 9/edge 14 10.246.118.59 no installing agent

Unit Workload Agent Address Ports Message
bind/0 blocked idle 10.1.0.202 (workload) Error in charm (see logs): bind/0 is not leader and cannot read its own application databag
bind/1 blocked idle 10.1.2.78 (workload) Error in charm (see logs): bind/1 is not leader and cannot read its own application databag
bind/2* active idle 10.1.1.91

Offer Application Charm Rev Connected Endpoint Interface Role
ceilometer ceilometer ceilometer-k8s 25 1/1 ceilometer-service ceilometer provider
cert-distributor keystone keystone-k8s 169 1/1 send-ca-cert certificate_transfer provider
certificate-authority certificate-authority self-signed-certificates 109 1/1 certificates tls-certificates provider
cinder-ceph cinder-ceph cinder-ceph-k8s 57 1/1 ceph-access cinder-ceph-key provider
keystone keystone keystone-k8s 169 1/1 identity-credentials keystone-credentials provider
ovn-relay ovn-relay ovn-relay-k8s 62 1/1 ovsdb-cms-relay ovsdb-cms provider
rabbitmq rabbitmq rabbitmq-k8s 34 1/1 amqp rabbitmq provider

Logs:
unit-bind-0: 11:16:22 ERROR unit.bind/0.juju-log peers:246: Exception raised in section 'Bootstrapping': bind/0 is not leader and cannot read its own application databag
unit-bind-0: 11:16:22 ERROR unit.bind/0.juju-log peers:246: Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/ops_sunbeam/guard.py", line 91, in guard
    yield
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/ops_sunbeam/charm.py", line 303, in configure_charm
    self.configure_unit(event)
  File "/var/lib/juju/agents/unit-bind-0/charm/./src/charm.py", line 249, in configure_unit
    self.init_container_services()
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/ops_sunbeam/charm.py", line 596, in init_container_services
    ph.init_service(self.contexts())
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/ops_sunbeam/charm.py", line 504, in contexts
    ra.add_relation_handler(handler)
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/ops_sunbeam/core.py", line 57, in add_relation_handler
    ctxt = handler.context()
  File "/var/lib/juju/agents/unit-bind-0/charm/./src/charm.py", line 179, in context
    rndc_keys = self.rndc_keys
  File "/var/lib/juju/agents/unit-bind-0/charm/./src/charm.py", line 157, in rndc_keys
    rndc_keys_secret = self.interface.get_rndc_keys(relation)
  File "/var/lib/juju/agents/unit-bind-0/charm/lib/charms/designate_bind_k8s/v0/bind_rndc.py", line 319, in get_rndc_keys
    return json.loads(relation.data[self.charm.app].get("rndc_keys", "{}"))
  File "/usr/lib/python3.10/_collections_abc.py", line 824, in get
    return self[key]
  File "/var/lib/juju/agents/unit-bind-0/charm/venv/ops/model.py", line 1705, in __getitem__
    self._validate_read()
  File "/var/lib/juju/agents/unit-bind-0/charm/venv/ops/model.py", line 1632, in _validate_read
    raise RelationDataAccessError(
ops.model.RelationDataAccessError: bind/0 is not leader and cannot read its own application databag

unit-bind-0: 11:16:22 INFO juju.worker.uniter.operation ran "peers-relation-changed" hook (via hook dispatching script: dispatch)
unit-bind-0: 11:16:27 INFO juju.worker.uniter.operation ran "dns-backend-relation-changed" hook (via hook dispatching script: dispatch)
unit-bind-0: 11:16:28 INFO juju.worker.uniter.operation ran "dns-backend-relation-changed" hook (via hook dispatching script: dispatch)
unit-bind-0: 11:16:29 INFO juju.worker.uniter.operation ran "dns-backend-relation-changed" hook (via hook dispatching script: dispatch)
unit-bind-0: 11:21:48 INFO unit.bind/0.juju-log HTTP Request: GET https://10.152.183.1/api/v1/namespaces/openstack/services/bind "HTTP/1.1 200 OK"
unit-bind-0: 11:21:48 INFO unit.bind/0.juju-log HTTP Request: PATCH https://10.152.183.1/api/v1/namespaces/openstack/services/bind "HTTP/1.1 200 OK"
unit-bind-0: 11:21:49 INFO unit.bind/0.juju-log Kubernetes service 'bind' patched successfully
unit-bind-0: 11:21:49 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)

Seems like this is just status update related issue.