Activity log for bug #1849624

Date Who What changed Old value New value Message
2019-10-24 08:45:29 raphael.glon bug added bug
2019-10-24 11:38:21 raphael.glon description Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "OpenStack-API-Version: volume 3.15" \ -H "X-Auth-Token: $TOKEN" \ -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. { "connection_info" : { "data" : { "access_mode" : "rw", "secret_uuid" : "SECRET_UUID", "cluster_name" : "ceph", "encrypted" : false, "auth_enabled" : true, "discard" : true, "qos_specs" : { "write_iops_sec" : "3050", "read_iops_sec" : "3050" }, "keyring" : "SECRETFILETOHIDE", "ports" : [ "6789", "6789", "6789" ], "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879", "secret_type" : "ceph", "hosts" : [ "ceph_host1", "ceph_host2", ... ], "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879", "auth_username" : "cinder" }, "driver_volume_type" : "rbd" } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, not harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \     -H "Accept: application/json" \     -H "Content-Type: application/json" \     -H "OpenStack-API-Version: volume 3.15" \     -H "X-Auth-Token: $TOKEN" \     -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. {    "connection_info" : {       "data" : {          "access_mode" : "rw",          "secret_uuid" : "SECRET_UUID",          "cluster_name" : "ceph",          "encrypted" : false,          "auth_enabled" : true,          "discard" : true,          "qos_specs" : {             "write_iops_sec" : "3050",             "read_iops_sec" : "3050"          },          "keyring" : "SECRETFILETOHIDE",          "ports" : [             "6789",             "6789",             "6789"          ],          "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879",          "secret_type" : "ceph",          "hosts" : [             "ceph_host1",             "ceph_host2",             ...          ],          "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879",          "auth_username" : "cinder"       },       "driver_volume_type" : "rbd"    } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one
2019-10-24 14:03:36 Sean McGinnis cinder: assignee Eric Harney (eharney)
2019-10-24 14:14:38 Jeremy Stanley bug task added ossa
2019-10-24 14:14:55 Jeremy Stanley ossa: status New Incomplete
2019-10-24 14:15:10 Jeremy Stanley description Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \     -H "Accept: application/json" \     -H "Content-Type: application/json" \     -H "OpenStack-API-Version: volume 3.15" \     -H "X-Auth-Token: $TOKEN" \     -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. {    "connection_info" : {       "data" : {          "access_mode" : "rw",          "secret_uuid" : "SECRET_UUID",          "cluster_name" : "ceph",          "encrypted" : false,          "auth_enabled" : true,          "discard" : true,          "qos_specs" : {             "write_iops_sec" : "3050",             "read_iops_sec" : "3050"          },          "keyring" : "SECRETFILETOHIDE",          "ports" : [             "6789",             "6789",             "6789"          ],          "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879",          "secret_type" : "ceph",          "hosts" : [             "ceph_host1",             "ceph_host2",             ...          ],          "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879",          "auth_username" : "cinder"       },       "driver_volume_type" : "rbd"    } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \     -H "Accept: application/json" \     -H "Content-Type: application/json" \     -H "OpenStack-API-Version: volume 3.15" \     -H "X-Auth-Token: $TOKEN" \     -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. {    "connection_info" : {       "data" : {          "access_mode" : "rw",          "secret_uuid" : "SECRET_UUID",          "cluster_name" : "ceph",          "encrypted" : false,          "auth_enabled" : true,          "discard" : true,          "qos_specs" : {             "write_iops_sec" : "3050",             "read_iops_sec" : "3050"          },          "keyring" : "SECRETFILETOHIDE",          "ports" : [             "6789",             "6789",             "6789"          ],          "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879",          "secret_type" : "ceph",          "hosts" : [             "ceph_host1",             "ceph_host2",             ...          ],          "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879",          "auth_username" : "cinder"       },       "driver_volume_type" : "rbd"    } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one
2019-10-24 14:15:25 Jeremy Stanley bug added subscriber Cinder Core security contacts
2019-10-30 13:22:19 Jeremy Stanley bug task added ossn
2019-10-30 13:22:30 Jeremy Stanley ossa: status Incomplete Won't Fix
2019-10-30 13:22:37 Jeremy Stanley ossn: status New Confirmed
2019-10-30 13:22:59 Jeremy Stanley ossn: assignee Brian Rosmaita (brian-rosmaita)
2019-10-30 13:23:21 Jeremy Stanley description This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \     -H "Accept: application/json" \     -H "Content-Type: application/json" \     -H "OpenStack-API-Version: volume 3.15" \     -H "X-Auth-Token: $TOKEN" \     -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. {    "connection_info" : {       "data" : {          "access_mode" : "rw",          "secret_uuid" : "SECRET_UUID",          "cluster_name" : "ceph",          "encrypted" : false,          "auth_enabled" : true,          "discard" : true,          "qos_specs" : {             "write_iops_sec" : "3050",             "read_iops_sec" : "3050"          },          "keyring" : "SECRETFILETOHIDE",          "ports" : [             "6789",             "6789",             "6789"          ],          "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879",          "secret_type" : "ceph",          "hosts" : [             "ceph_host1",             "ceph_host2",             ...          ],          "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879",          "auth_username" : "cinder"       },       "driver_volume_type" : "rbd"    } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one Cinder + ceph backend, secret key leak Conditions: cinder + ceph backend + rbd_keyring_conf set in cinder config files As an authenticated simple user create a cinder volume that ends up on a ceph backend, Then reuse the os.initialize_connection api call (used by nova-compute/cinder-backup to attach volumes locally to the host running the services): curl -g -i -X POST https://<cinder_controller>/v3/c495530af57611e9bc14bbaa251e1e96/volumes/7e59b91e-d426-4294-bfc5-dfdebcb21879/action \     -H "Accept: application/json" \     -H "Content-Type: application/json" \     -H "OpenStack-API-Version: volume 3.15" \     -H "X-Auth-Token: $TOKEN" \     -d '{"os-initialize_connection": {"connector":{}}}' If you do not want to forge the http request, openstack clients and extensions may prove helpful. As root: apt-get install python3-oslo.privsep virtualenv python3-dev python3-os-brick gcc ceph-common virtualenv -p python3 venv_openstack source venv_openstack/bin/activate pip install python-openstackclient pip install python-cinderclient pip install os-brick pip install python-brick-cinderclient-ext cinder create vol 1 cinder --debug local-attach 7e59b91e-d426-4294-bfc5-dfdebcb21879 This leaks the ceph credentials for the whole ceph cluster, leaving anyone able to go through ceph acls to get access to all the volumes within the cluster. {    "connection_info" : {       "data" : {          "access_mode" : "rw",          "secret_uuid" : "SECRET_UUID",          "cluster_name" : "ceph",          "encrypted" : false,          "auth_enabled" : true,          "discard" : true,          "qos_specs" : {             "write_iops_sec" : "3050",             "read_iops_sec" : "3050"          },          "keyring" : "SECRETFILETOHIDE",          "ports" : [             "6789",             "6789",             "6789"          ],          "name" : "volumes/volume-7e59b91e-d426-4294-bfc5-dfdebcb21879",          "secret_type" : "ceph",          "hosts" : [             "ceph_host1",             "ceph_host2",             ...          ],          "volume_id" : "7e59b91e-d426-4294-bfc5-dfdebcb21879",          "auth_username" : "cinder"       },       "driver_volume_type" : "rbd"    } } Quick workaround: 1. Remove rbd_keyring_conf param from any cinder config file, this will mitigate the information disclosure. 2. For cinder backups to still work, providers should instead deploy their ceph keyring secrets directly on cinder-backup hosts (/etc/cinder/<backend_name>.keyring.conf, to be confirmed). Note that nova-compute hosts should not be impacted by the change, because ceph secrets are expected to be stored in libvirt secrets already, thus making this keyring disclose useless to it. (to be confirmed, there may be other compute drivers that might be impacted by this) Quick code fix: Mandatory: revert this commit https://review.opendev.org/#/c/456672/ Optional: revert this one https://review.opendev.org/#/c/465044/, harmless in itself, but pointless once the first one has been reverted Long term code fix proposals: What the os.initialize_connection api call is meant to: allow simple users to use cinder as block storage as a service in order to attach volumes outside the scope of any virtual machines/nova. Thus, information returned by this call should give enough information for a volume attach to be possible for the caller but they should not disclose anything that would allow him to do more than that. Since it is not possible at all with ceph to do so (no tenant isolation within ceph cluster), the related cinder backend for ceph should not implement this route at all There is indeed no reason why cinder should disclose anything here about ceph cluster, including hosts, cluster-ids, if the attach is doomed to fail for users missing secret informations anyway. Then, any 'admin' service using this call to locally attach the volumes (nova-compute, cinder-backup...) should be modified to: - check caller rw permissions on requested volumes - escalate the request - go through a new admin api route, not this 'user' one
2019-10-30 13:23:27 Jeremy Stanley information type Private Security Public
2019-10-30 13:23:41 Jeremy Stanley tags ceph ceph security
2019-10-30 14:56:57 Brian Rosmaita cinder: importance Undecided Low
2019-10-30 14:56:57 Brian Rosmaita cinder: status New In Progress
2019-10-30 14:56:57 Brian Rosmaita cinder: milestone ussuri-1
2019-10-30 14:56:57 Brian Rosmaita cinder: assignee Eric Harney (eharney) Brian Rosmaita (brian-rosmaita)
2019-10-31 15:28:12 Gage Hugo ossn: status Confirmed In Progress
2019-11-20 21:27:27 Brian Rosmaita ossn: status In Progress Fix Released
2019-11-20 21:27:51 Brian Rosmaita cinder: status In Progress Fix Committed
2019-11-21 13:31:11 Brian Rosmaita cinder: status Fix Committed In Progress
2019-11-21 19:25:17 Brian Rosmaita cinder: status In Progress Fix Committed
2020-05-13 20:34:56 OpenStack Infra tags ceph security ceph in-stable-ussuri security
2020-05-14 00:13:01 OpenStack Infra tags ceph in-stable-ussuri security ceph in-stable-train in-stable-ussuri security
2020-05-14 17:04:15 OpenStack Infra tags ceph in-stable-train in-stable-ussuri security ceph in-stable-stein in-stable-train in-stable-ussuri security
2020-05-27 15:26:15 OpenStack Infra tags ceph in-stable-stein in-stable-train in-stable-ussuri security ceph in-stable-rocky in-stable-stein in-stable-train in-stable-ussuri security