There is a bug with the service token configuration when cinder is separated between cinder and cinder-volume. Plain cider uses the identity-service relation with keystone while cinder-volume uses identity-credentials. The token created with the credentials obtained from identity-credentials fails to authenticate on keystone. I can reproduce this on a focal/yoga openstack installation using cinder-volume and cinder-ceph but it also happens with pure-storage. Using the default service token configuration cinder-volume can not create a volume based on an image because glance rejects the service token. This can be fixed by removing the [service-user] section on cinder-volume or by using the same credentials as those obtained by the identity-service relation. This is the exception shown on the cinder-volume.log: 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server [req-a49bac91-585d-47bd-9ed2-5a858f4ca2de 1d0e71f8854a4cddab77ac23b88dd841 38560b4f0be54e17af9a90ca2e90144f - - -] Exception during message handling: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f0458e09-ee73-4506-b053-f7b6553f6317) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "", line 2, in create_volume 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/objects/cleanable.py", line 208, in wrapper 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 815, in create_volume 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server _run_flow() 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 807, in _run_flow 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server flow_engine.run() 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 339, in reraise_if_any 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server failures[0].reraise() 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/types/failure.py", line 346, in reraise 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise value 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/volume/flows/manager/create_volume.py", line 322, in execute 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server 'image_location': image_service.get_location(context, 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 336, in get_location 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server _reraise_translated_image_exception(image_id) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 635, in _reraise_translated_image_exception 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise new_exc.with_traceback(exc_trace) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 334, in get_location 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server image_meta = client.call(context, 'get', image_id) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/image/glance.py", line 236, in call 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return getattr(controller, method)(*args, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/v2/images.py", line 197, in get 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._get(image_id) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/common/utils.py", line 670, in inner 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return RequestIdProxy(wrapped(*args, **kwargs)) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/v2/images.py", line 190, in _get 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp, body = self.http_client.get(url, headers=header) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 395, in get 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.request(url, 'GET', **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/glanceclient/common/http.py", line 361, in request 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp = super(SessionClient, 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/adapter.py", line 257, in request 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.session.request(url, method, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 790, in request 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server auth_headers = self.get_auth_headers(auth) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1201, in get_auth_headers 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return auth.get_headers(self, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/service_token.py", line 29, in get_headers 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server token = self.service_auth.get_token(session, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 88, in get_token 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.get_access(session).auth_token 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server self.auth_ref = self.get_auth_ref(session) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self._plugin.get_auth_ref(session, **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/identity/v3/base.py", line 187, in get_auth_ref 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server resp = session.post(token_url, json=body, headers=headers, 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 1149, in post 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server return self.request(url, 'POST', **kwargs) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/keystoneauth1/session.py", line 986, in request 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server raise exceptions.from_response(resp, method, url) 2023-06-22 14:24:28.231 544373 ERROR oslo_messaging.rpc.server keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-f0458e09-ee73-4506-b053-f7b6553f6317)