This is the failure in the compute during the unrescue:
http://paste.openstack.org/show/570328/
2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [req-2bd83902-baec-42cf-824d-d1c16aaedd4f tempest-ServerRescueNegativeTestJSON-727441625 tempest-ServerRescueNegativeTestJSON-727441625] [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] Setting instance vm_state to ERROR 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] Traceback (most recent call last): 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/compute/manager.py", line 6432, in _error_out_instance_on_exception 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] yield 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/compute/manager.py", line 3262, in unrescue_instance 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] network_info) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2544, in unrescue 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] self._create_domain(xml, virt_dom) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4650, in _create_domain 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] guest.launch(pause=pause) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 142, in launch 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] self._encoded_xml, errors='ignore') 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] self.force_reraise() 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] six.reraise(self.type_, self.value, self.tb) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 137, in launch 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] return self._domain.createWithFlags(flags) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] result = proxy_call(self._autowrap, f, *args, **kwargs) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] rv = execute(f, *args, **kwargs) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] six.reraise(c, e, tb) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] rv = meth(*args, **kwargs) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 900, in createWithFlags 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244] libvirtError: Failed to open file '/mnt/quobyte-volume/abfa1002557ab2b21ec218a86487dd92/volume-351db2c5-9724-410f-b1d8-8680065c0788': No such file or directory 2016-07-23 13:03:19.943 1365 ERROR nova.compute.manager [instance: 35a2d53a-d80d-46b2-87a6-a7a82f9d5244]
This is the failure in the compute during the unrescue:
http:// paste.openstack .org/show/ 570328/
2016-07-23 13:03:19.943 1365 ERROR nova.compute. manager [req-2bd83902- baec-42cf- 824d-d1c16aaedd 4f tempest- ServerRescueNeg ativeTestJSON- 727441625 tempest- ServerRescueNeg ativeTestJSON- 727441625] [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] Setting instance vm_state to ERROR manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] Traceback (most recent call last): manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ compute/ manager. py", line 6432, in _error_ out_instance_ on_exception manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] yield manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ compute/ manager. py", line 3262, in unrescue_instance manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] network_info) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ virt/libvirt/ driver. py", line 2544, in unrescue manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] self._create_ domain( xml, virt_dom) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ virt/libvirt/ driver. py", line 4650, in _create_domain manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] guest.launch( pause=pause) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ virt/libvirt/ guest.py" , line 142, in launch manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] self._encoded_xml, errors='ignore') manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ oslo_utils/ excutils. py", line 220, in __exit__ manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] self.force_ reraise( ) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ oslo_utils/ excutils. py", line 196, in force_reraise manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] six.reraise( self.type_ , self.value, self.tb) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/opt/stack/ nova/nova/ virt/libvirt/ guest.py" , line 137, in launch manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] return self._domain. createWithFlags (flags) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ eventlet/ tpool.py" , line 186, in doit manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] result = proxy_call( self._autowrap, f, *args, **kwargs) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ eventlet/ tpool.py" , line 144, in proxy_call manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] rv = execute(f, *args, **kwargs) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ eventlet/ tpool.py" , line 125, in execute manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] six.reraise(c, e, tb) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ eventlet/ tpool.py" , line 83, in tworker manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] rv = meth(*args, **kwargs) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] File "/usr/local/ lib/python2. 7/dist- packages/ libvirt. py", line 900, in createWithFlags manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] if ret == -1: raise libvirtError ('virDomainCrea teWithFlags( ) failed', dom=self) manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44] libvirtError: Failed to open file '/mnt/quobyte- volume/ abfa1002557ab2b 21ec218a86487dd 92/volume- 351db2c5- 9724-410f- b1d8-8680065c07 88': No such file or directory manager [instance: 35a2d53a- d80d-46b2- 87a6-a7a82f9d52 44]
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.
2016-07-23 13:03:19.943 1365 ERROR nova.compute.