cinder-ceph unit status update is not real time

[Hemanth Nakkina]

Deployed sunbeam 2023.2/edge rev 465.
Bootstrapped single node with roles control, compute, storage.

cinder-ceph application is not active for long time. Seems like the status gets updated to active during update-status hook.

Juju status:
$ juju status -m openstack cinder-ceph
Model Controller Cloud/Region Version SLA Timestamp
openstack sunbeam-controller sunbeam-k8s/localhost 3.4.1 unsupported 02:10:39Z

SAAS Status Store URL
microceph active local admin/controller.microceph

App Version Status Scale Charm Channel Rev Address Exposed Message
cinder-ceph waiting 1 cinder-ceph-k8s 2023.2/edge 57 10.152.183.200 no installing agent

Unit Workload Agent Address Ports Message
cinder-ceph/0* waiting idle 10.1.0.120 (container:cinder-volume) service not ready

Offer Application Charm Rev Connected Endpoint Interface Role
cert-distributor keystone keystone-k8s 169 0/0 send-ca-cert certificate_transfer provider
certificate-authority certificate-authority self-signed-certificates 109 0/0 certificates tls-certificates provider
cinder-ceph cinder-ceph cinder-ceph-k8s 57 0/0 ceph-access cinder-ceph-key provider
keystone keystone keystone-k8s 169 0/0 identity-credentials keystone-credentials provider
ovn-relay ovn-relay ovn-relay-k8s 62 0/0 ovsdb-cms-relay ovsdb-cms provider
rabbitmq rabbitmq rabbitmq-k8s 34 0/0 amqp rabbitmq provide

Pebble shows service is active
$ sudo k8s kubectl -n openstack exec -it cinder-ceph-0 -c cinder-volume -- /bin/bash
root@cinder-ceph-0:/# pebble services
Service Startup Current Since
cinder-volume disabled active today at 02:08 UTC

Juju debug logs:

unit-cinder-ceph-0: 02:03:00 INFO unit.cinder-ceph/0.juju-log peers:35: Entering guarded section: 'Bootstrapping'
unit-cinder-ceph-0: 02:03:08 WARNING unit.cinder-ceph/0.juju-log peers:35: Not running db-sync, it has run previously for this unit
unit-cinder-ceph-0: 02:03:08 INFO unit.cinder-ceph/0.juju-log peers:35: Setting active status
unit-cinder-ceph-0: 02:03:08 INFO unit.cinder-ceph/0.juju-log peers:35: Completed guarded section fully: 'Bootstrapping'
unit-cinder-ceph-0: 02:03:08 INFO unit.cinder-ceph/0.juju-log peers:35: CONFIG changed and ceph ready: calling request pools
unit-cinder-ceph-0: 02:03:09 INFO unit.cinder-ceph/0.juju-log peers:35: add_op_create_replicated_pool: [<ops.model.Relation ceph:73>]
unit-cinder-ceph-0: 02:03:09 INFO unit.cinder-ceph/0.juju-log peers:35: Adding add_op_create_replicated_pool request
unit-cinder-ceph-0: 02:03:09 INFO unit.cinder-ceph/0.juju-log peers:35: Storing request
unit-cinder-ceph-0: 02:03:09 INFO unit.cinder-ceph/0.juju-log peers:35: Calling send_request_if_needed
unit-cinder-ceph-0: 02:03:10 INFO juju.worker.uniter.operation ran "peers-relation-changed" hook (via hook dispatching script: dispatch)
unit-cinder-ceph-0: 02:05:33 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)
unit-cinder-ceph-0: 02:07:49 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Entering guarded section: 'Bootstrapping'
unit-cinder-ceph-0: 02:08:04 WARNING unit.cinder-ceph/0.juju-log storage-backend:74: Not running db-sync, it has run previously for this unit
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Setting active status
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Completed guarded section fully: 'Bootstrapping'
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: CONFIG changed and ceph ready: calling request pools
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: add_op_create_replicated_pool: [<ops.model.Relation ceph:73>]
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Adding add_op_create_replicated_pool request
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Storing request
unit-cinder-ceph-0: 02:08:05 INFO unit.cinder-ceph/0.juju-log storage-backend:74: Calling send_request_if_needed
unit-cinder-ceph-0: 02:08:08 INFO juju.worker.uniter.operation ran "storage-backend-relation-changed" hook (via hook dispatching script: dispatch)
unit-cinder-ceph-0: 02:11:29 INFO juju.worker.uniter.operation ran "update-status" hook (via hook dispatching script: dispatch)

After storage-backend-relation-changed hook is handled, the unit status is not active until last statement update-status hook.