cinder nova volume attach not working

[Jeff Peeler]

log/debug provided by Ryan:

2015-06-26 21:46:19.148 1 INFO nova.compute.manager [req-da63d4c6-579e-416f-a6fa-cd246074df3e 5759e5c1587945c7a74d6a481285a580 102baa7f2b8749aba7f1cc80fe4520f4 - - -] [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Attaching volume 031f9e4b-d03c-4847-bbf9-649827e5b540 to /dev/vdb
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [req-da63d4c6-579e-416f-a6fa-cd246074df3e 5759e5c1587945c7a74d6a481285a580 102baa7f2b8749aba7f1cc80fe4520f4 - - -] [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Driver failed to attach volume 031f9e4b-d03c-4847-bbf9-649827e5b540 at /dev/vdb
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Traceback (most recent call last):
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 254, in attach
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] device_type=self['device_type'], encryption=encryption)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1050, in attach_volume
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] self._connect_volume(connection_info, disk_info)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1001, in _connect_volume
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] driver.connect_volume(connection_info, disk_info)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] return f(*args, **kwargs)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 434, in connect_volume
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] self._run_iscsiadm(iscsi_properties, ("--rescan",))
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 359, in _run_iscsiadm
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] check_exit_code=check_exit_code)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/utils.py", line 228, in execute
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] return processutils.execute(*cmd, **kwargs)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 240, in execute
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] cmd=sanitized_cmd)
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] ProcessExecutionError: Unexpected error while running command.
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Command: iscsiadm -m node -T iqn.2010-10.org.openstack:volume-031f9e4b-d03c-4847-bbf9-649827e5b540 -p 10.8.144.14:3260 --rescan
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Exit code: 21
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Stdout: u''
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Stderr: u'iscsiadm: No session found.\n'
2015-06-26 21:46:22.770 1 ERROR nova.virt.block_device [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8]
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [req-da63d4c6-579e-416f-a6fa-cd246074df3e 5759e5c1587945c7a74d6a481285a580 102baa7f2b8749aba7f1cc80fe4520f4 - - -] [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Failed to attach 031f9e4b-d03c-4847-bbf9-649827e5b540 at /dev/vdb
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Traceback (most recent call last):
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4443, in _attach_volume
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] do_check_attach=False, do_driver_attach=True)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 50, in wrapped
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] ret_val = method(obj, context, *args, **kwargs)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 263, in attach
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] connector)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 254, in attach
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] device_type=self['device_type'], encryption=encryption)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1050, in attach_volume
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] self._connect_volume(connection_info, disk_info)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1001, in _connect_volume
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] driver.connect_volume(connection_info, disk_info)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] return f(*args, **kwargs)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 434, in connect_volume
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] self._run_iscsiadm(iscsi_properties, ("--rescan",))
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 359, in _run_iscsiadm
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] check_exit_code=check_exit_code)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/nova/utils.py", line 228, in execute
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] return processutils.execute(*cmd, **kwargs)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 240, in execute
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] cmd=sanitized_cmd)
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] ProcessExecutionError: Unexpected error while running command.
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Command: iscsiadm -m node -T iqn.2010-10.org.openstack:volume-031f9e4b-d03c-4847-bbf9-649827e5b540 -p 10.8.144.14:3260 --rescan
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Exit code: 21
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Stdout: u''
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8] Stderr: u'iscsiadm: No session found.\n'
2015-06-26 21:46:23.352 1 ERROR nova.compute.manager [instance: 09e5d23c-cfd9-41a9-8721-331b942efea8]
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher [req-da63d4c6-579e-416f-a6fa-cd246074df3e 5759e5c1587945c7a74d6a481285a580 102baa7f2b8749aba7f1cc80fe4520f4 - - -] Exception during message handling: Unexpected error while running command.
Command: iscsiadm -m node -T iqn.2010-10.org.openstack:volume-031f9e4b-d03c-4847-bbf9-649827e5b540 -p 10.8.144.14:3260 --rescan
Exit code: 21
Stdout: u''
Stderr: u'iscsiadm: No session found.\n'
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher payload)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 335, in decorated_function
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 306, in decorated_function
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 363, in decorated_function
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info())
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 351, in decorated_function
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4433, in attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher do_attach_volume(context, instance, driver_bdm)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4431, in do_attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher bdm.destroy()
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4428, in do_attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return self._attach_volume(context, instance, driver_bdm)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4451, in _attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher self.volume_api.unreserve_volume(context, bdm.volume_id)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4443, in _attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher do_check_attach=False, do_driver_attach=True)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 50, in wrapped
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher ret_val = method(obj, context, *args, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 263, in attach
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher connector)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 119, in __exit__
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 254, in attach
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher device_type=self['device_type'], encryption=encryption)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1050, in attach_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher self._connect_volume(connection_info, disk_info)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1001, in _connect_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher driver.connect_volume(connection_info, disk_info)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 445, in inner
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 434, in connect_volume
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher self._run_iscsiadm(iscsi_properties, ("--rescan",))
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 359, in _run_iscsiadm
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher check_exit_code=check_exit_code)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/utils.py", line 228, in execute
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher return processutils.execute(*cmd, **kwargs)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 240, in execute
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher cmd=sanitized_cmd)
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher ProcessExecutionError: Unexpected error while running command.
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher Command: iscsiadm -m node -T iqn.2010-10.org.openstack:volume-031f9e4b-d03c-4847-bbf9-649827e5b540 -p 10.8.144.14:3260 --rescan
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher Exit code: 21
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher Stdout: u''
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher Stderr: u'iscsiadm: No session found.\n'
2015-06-26 21:46:23.553 1 ERROR oslo_messaging.rpc.dispatcher

[Steven Dake (sdake)]

While it would be nice if this problem could be solved prior to Monday, this won't be a blocker for our release. We are releasing Monday, assuming the dhcp issue with neutron can be resolved.

[Steven Dake (sdake)]

More data from cinder-volume container:
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 252, in inner
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio return f(*args, **kwargs)
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/volume/targets/lio.py", line 70, in _execute
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio return utils.execute(*args, **kwargs)
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 152, in execute
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio return processutils.execute(*cmd, **kwargs)
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 275, in execute
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio cmd=sanitized_cmd)
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio ProcessExecutionError: Unexpected error while running command.
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio Command: cinder-rtstool create /dev/cinder-volumes/volume-1a41e8bc-de7f-4ec0-acaf-9661c07609c0 iqn.2010-10.org.openstack:volume-1a41e8bc-de7f-4ec0-acaf-9661c07609c0 CVDzf5gZa8W4nh9enmfR Ls4sJGQVvjCHp2Y4 False -p3260 -a192.168.1.100
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio Exit code: 1
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio Stdout: u''
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio Stderr: u'Traceback (most recent call last):\n File "/usr/bin/cinder-rtstool", line 10, in <module>\n sys.exit(main())\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 271, in main\n **optional_args)\n File "/usr/lib/python2.7/site-packages/cinder/cmd/rtstool.py", line 97, in create\n % {\'msg_type\': msg_type, \'port\': portals_port, \'ip\': ip})\n File "/usr/lib/python2.7/site-packages/oslo_i18n/_message.py", line 208, in __str__\n raise UnicodeError(msg)\nUnicodeError\n'
2015-07-10 19:41:45.793 98 ERROR cinder.volume.targets.lio
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager [req-d2479d31-1688-412d-8044-465ffee9137f 0823a3ed7aff4dd1b73c1213cc6747d6 39e82b5267dd4456ba4641f18752ee6e - - -] Create export for volume failed.
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager Traceback (most recent call last):
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1082, in initialize_connection
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager volume)
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager return f(*args, **kwargs)
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 643, in create_export
2015-07-10 19:41:45.795 98 ERROR cinder.volume.manager volume_path)
2015-07-10 19:41:45....

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/200139
Committed: https://git.openstack.org/cgit/stackforge/kolla/commit/?id=4008ca8619a83ac4ee9c56cb37f42b208d753823
Submitter: Jenkins
Branch: master

commit 4008ca8619a83ac4ee9c56cb37f42b208d753823
Author: Steven Dake <email address hidden>
Date: Thu Jul 9 07:49:28 2015 -0700

    Start tgtd in cinder-volume container

    The tgtd process was not being started in the cinder-volume container.
    This results in iSCSI not being avialable for Cinder to operate
    against. Unfortunately cinder doesn't report the problem because LVM
    is responsible for the iSCSI mapping.

    Co-Authored-By: Ryan Hallisey <email address hidden>

    Closes-Bug: 1473090
    Closes-Bug: 1469323
    Change-Id: Ie3403c5c3f81beb48960e838b4274e0c5796e387