| 2020-02-20 07:01:49 |
Abhishek Kekane |
description |
Image upload is failing with NoFibreChannelVolumeDeviceFound error after configuring cinder backend (HP3Par FC storage) for the glance service.
The issue here we are facing is that usually cinder calls os_brick with multipath info but in this case glance is doing the call (without passing any multipath info). Cinder driver of glance_store passes 'hard-coded' False as multipath is not used to os_brick library.
In order to fix this we need to add new config option for cinder driver of glance_store 'multipath_enabled' True|False to avoid such issues.
Steps to reproduce:
1. Configure cinder back end(HP3Par FC storage) for the Image service
2. Create image
$ glance image-create --name "cirros2" --file cirros-0.3.0-i386-disk.img --disk-format qcow2 --container-format bare
Output:
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
glance-api.log:-
~~~
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi Traceback (most recent call last):
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1227, in __call__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi request, **action_args)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1270, in dispatch
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return method(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 414, in wrapped
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 278, in upload
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self._restore(image_repo, image)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.force_reraise()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 143, in upload
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi image.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.base.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 480, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.force_reraise()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 427, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.repo.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 202, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.image.set_data(data, size=size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 439, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier=verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 453, in add_to_backend
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 426, in store_add_to_backend
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier=verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/cinder.py", line 677, in add
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi with self._open_cinder_volume(client, volume, 'wb') as f:
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.gen.next()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/cinder.py", line 497, in _open_cinder_volume
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi device = conn.connect_volume(connection_info['data'])
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi result = f(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 274, in inner
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return f(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 245, in connect_volume
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi timer.start(interval=2).wait()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return hubs.get_hub().switch()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.greenlet.switch()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 137, in _run_loop
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi result = func(*self.args, **self.kw)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 231, in _wait_for_device_discovery
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi raise exception.NoFibreChannelVolumeDeviceFound()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume
~~~ |
Image upload is failing with NoFibreChannelVolumeDeviceFound error after configuring cinder backend (HP3Par FC storage) for the glance service.
The issue here we are facing is that usually cinder calls os_brick with multipath info but in this case glance is doing the call (without passing any multipath info). Cinder driver of glance_store passes 'hard-coded' False as multipath is not used to os_brick library.
In order to fix this we need to add new config option for cinder driver of glance_store 'cinder_use_multipath' True|False to avoid such issues.
Steps to reproduce:
1. Configure cinder back end(HP3Par FC storage) for the Image service
2. Create image
$ glance image-create --name "cirros2" --file cirros-0.3.0-i386-disk.img --disk-format qcow2 --container-format bare
Output:
500 Internal Server Error: The server has either erred or is incapable of performing the requested operation. (HTTP 500)
glance-api.log:-
~~~
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi Traceback (most recent call last):
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1227, in __call__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi request, **action_args)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1270, in dispatch
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return method(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 414, in wrapped
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 278, in upload
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self._restore(image_repo, image)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.force_reraise()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 143, in upload
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi image.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.base.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 480, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.force_reraise()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 427, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.repo.set_data(data, size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 202, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi self.image.set_data(data, size=size)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 439, in set_data
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier=verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 453, in add_to_backend
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 426, in store_add_to_backend
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi verifier=verifier)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/cinder.py", line 677, in add
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi with self._open_cinder_volume(client, volume, 'wb') as f:
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.gen.next()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/cinder.py", line 497, in _open_cinder_volume
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi device = conn.connect_volume(connection_info['data'])
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi result = f(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 274, in inner
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return f(*args, **kwargs)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 245, in connect_volume
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi timer.start(interval=2).wait()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return hubs.get_hub().switch()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi return self.greenlet.switch()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_service/loopingcall.py", line 137, in _run_loop
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi result = func(*self.args, **self.kw)
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/fibre_channel.py", line 231, in _wait_for_device_discovery
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi raise exception.NoFibreChannelVolumeDeviceFound()
2019-12-24 07:10:30.869 28 ERROR glance.common.wsgi NoFibreChannelVolumeDeviceFound: Unable to find a Fibre Channel volume
~~~ |
|
