Comment 2 for bug 1928690

I tried this in nova_compute:

(nova-compute)[root@chrnc-void-testupgrade-compute-1-replace /]# ceph -n client.admin --keyring /var/lib/kolla/config_files/ceph.client.nova.keyring health
[errno 13] error connecting to the cluster

But I get the same result on a working cluster:

(nova-compute)[root@anaca-os-compute-01 /]# ceph -n client.admin --keyring /var/lib/kolla/config_files/ceph.client.nova.keyring health
[errno 13] error connecting to the cluster

I checked perms:

client.nova
        key: AQACHe1eAAAAABAAEytR6/2A/5y7O7/C/mjldw==
        caps: [mon] profile rbd
        caps: [osd] profile rbd pool=vms

It looks like the nova client can only run rbd commands, but rbd commands also fail on the working cluster:

(nova-compute)[root@anaca-os-compute-01 /]# rbd --keyring /var/lib/kolla/config_files/ceph.client.nova.keyring ls
2021-05-19 16:52:44.364 7f9f929fa700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-05-19 16:52:44.364 7f9f899f8700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-05-19 16:52:44.364 7f9f921f9700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
rbd: couldn't connect to the cluster!
rbd: listing images failed: (13) Permission denied

What am I missing?