client keyring not getting updated

[Harry Kominos]

Description
===========
When deploying on Wallaby with an external changing the cephclientkey does not update the keyring.

Environment
===========
openstack overcloud deploy --templates \ --skip-nodes-and-networks \ -e /home/stack/templates/environments/deployed-server-environment.yaml \ -r /home/stack/templates/roles_data.yaml \ -n /home/stack/templates/network_data.yaml \ -e...... -e /home/stack/templates/environments/external-ceph.yaml \ -e /home/stack/templates/novafixes.yaml \ ... --timeout 1500 --disable-validations --stack overcloud --rm-heat --deployed-server

Is the installation with wallaby, Centos8-stream, ovs, and external ceph.

with external-ceph.yaml containing a value for CephClusterFSID and CephClientKey.

If for some reason we would decide to use another ceph cluster (or recreate) and we would need
a new CephClusterFSID and a new CephClientKey, then substituting the new values leads all the contaienrs still having the old config in their /etc/ceph directory, which makes the deploypass but hte underlying openstack does not work.

Steps to reproduce
===============
1)deploy with external ceph
2)change cephID and cephclientkey
3)redeploy.
4)openstack image list (for example).

The following is also another way to see.
podman exec -it -u root nova_libvirt /usr/bin/cat /etc/ceph/ceph.client.openstack.keyring --> does not change
(possibly does not apply for ceph.conf which DOES get updated)

Expected result