[nova][cinder SVC driver][rhel7] The missing packages(sysfsutils sg3_utils) in nova-compute node cause nova volume-attach command failed

[Lan Qi song]

When I attached a SVC cinder volume to an instance, in nova-compute log there was a following error:

2015-01-19 15:54:34.913 2681 ERROR oslo.messaging._drivers.common [req-98e3cc7d-afd9-4348-b561-bef704cbe889 ] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n result = getattr(endpoint, method)(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 415, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped\n payload)\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 299, in decorated_function\n pass\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 284, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 327, in decorated_function\n kwargs[\'instance\'], e, sys.exc_info())\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 315, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4660, in attach_volume\n do_attach_volume(context, instance, driver_bdm)\n', ' File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 431, in inner\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4658, in do_attach_volume\n bdm.destroy(context)\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4655, in do_attach_volume\n return self._attach_volume(context, instance, driver_bdm)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4678, in _attach_volume\n self.volume_api.unreserve_volume(context, bdm.volume_id)\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4670, in _attach_volume\n do_check_attach=False, do_driver_attach=True)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 48, in wrapped\n ret_val = method(obj, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 263, in attach\n connector)\n', ' File "/usr/lib/python2.7/site-packages/oslo/utils/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 254, in attach\n device_type=self[\'device_type\'], encryption=encryption)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1077, in attach_volume\n self._connect_volume(connection_info, disk_info)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1024, in _connect_volume\n driver.connect_volume(connection_info, disk_info)\n', ' File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 431, in inner\n return f(*args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 1089, in connect_volume\n raise exception.NovaException(msg)\n', 'NovaException: We are unable to locate any Fibre Channel devices\n']

Then, I installed the "sysfsutils" in nova-compute node through # yum install -y sysfsutils
and attached again, still failed:

Command: sudo nova-rootwrap /etc/nova/rootwrap.conf sg_scan /dev/sdg
Exit code: 96
Stdout: u''
Stderr: u'/usr/bin/nova-rootwrap: Executable not found: sg_scan (filter match = sg_scan)\n'

Finally, I installed the "sg3_utils" packages in nova-compute node, the nova volume-attach function worked fine.

I think we should modify the recipe to install "sysfsutils" and 'sg3_utils' packages in nova-compute node.

[Mark Vanderwiel (vanderwl)]

Humm, trying to understand this one better. From the block storage cookbook, I see https://github.com/stackforge/cookbook-openstack-block-storage/blob/master/attributes/default.rb#L324 where we install the sysfsutils package.
But I don't see sg3_utils anywhere in block storage. Is this just needed on the compute side?

[Mark Vanderwiel (vanderwl)]

Ah, found the doc here for the required packages: http://docs.openstack.org/juno/config-reference/content/fibre-channel-packages.html

So, final question, do we just always install these, or gate that on some other attribute/condition?

[Lan Qi song (lqslan)]

sg3_utils packages are only needed on the compute side for initialize the FC device. The best way is to install the packages only when the block storage driver set to SVC, but it seems no that easy to judge the block storage driver type in compute cookbook.

[Mark Vanderwiel (vanderwl)]

Addressed by https://review.openstack.org/#/c/159265/