volume attach test race fails in dsvm-tempest-xen CI due to: "libvirtError: internal error: libxenlight failed to attach disk 'xvdb'"

Bug #1518007 reported by Matt Riedemann on 2015-11-19
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
Unassigned

Bug Description

Seen in a CI run here:

http://logs.openstack.xenproject.org/79/241179/9/check/dsvm-tempest-xen/6901c94/logs/screen-n-cpu.txt.gz

This was unrelated to the change being tested: https://review.openstack.org/#/c/241179/

2015-11-19 05:41:56.395 ERROR nova.compute.manager [req-0f579e1a-f928-4f24-8beb-8288a2699284 tempest-AttachVolumeTestJSON-174551048 tempest-AttachVolumeTestJSON-1798726384] [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] Failed to attach 853230d9-8e71-416b-a27b-0107ca181020 at /dev/xvdb
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] Traceback (most recent call last):
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/compute/manager.py", line 4653, in _attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] do_check_attach=False, do_driver_attach=True)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 56, in wrapped
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] ret_val = method(obj, context, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 297, in attach
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] connector)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(self.type_, self.value, self.tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 288, in attach
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] device_type=self['device_type'], encryption=encryption)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1123, in attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] self._disconnect_volume(connection_info, disk_dev)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(self.type_, self.value, self.tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1112, in attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] guest.attach_device(conf, persistent=True, live=live)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 233, in attach_device
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] self._domain.attachDeviceFlags(conf.to_xml(), flags=flags)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] result = proxy_call(self._autowrap, f, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] rv = execute(f, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(c, e, tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] rv = meth(*args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 554, in attachDeviceFlags
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] libvirtError: internal error: libxenlight failed to attach disk 'xvdb'
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d]

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers