Here is the pretty obvious failure in the cells v1 logs:
http://logs.openstack.org/06/425806/8/check/gate-tempest-dsvm-cells-ubuntu-xenial/ab0bba2/logs/screen-n-cell-child.txt.gz?level=TRACE#_2017-01-31_01_25_10_115
2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging [req-a820f01f-ff4d-4c62-8a71-ba509d2061aa tempest-TestVolumeSwap-893937839 tempest-TestVolumeSwap-893937839] Error processing message locally 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging Traceback (most recent call last): 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/cells/messaging.py", line 188, in _process_locally 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging resp_value = self.msg_runner._process_message_locally(self) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/cells/messaging.py", line 1275, in _process_message_locally 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return fn(message, **message.method_kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/cells/messaging.py", line 677, in run_compute_api_method 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return fn(message.ctxt, *args, **method_info['method_kwargs']) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/compute/api.py", line 170, in inner 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return function(self, context, instance, *args, **kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/compute/api.py", line 151, in inner 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return f(self, context, instance, *args, **kw) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/compute/api.py", line 3697, in detach_volume 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging self._detach_volume(context, instance, volume) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/compute/api.py", line 3665, in _detach_volume 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging self._check_and_begin_detach(context, volume, instance) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/compute/api.py", line 3657, in _check_and_begin_detach 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging self.volume_api.begin_detaching(context, volume['id']) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/volume/cinder.py", line 176, in wrapper 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging _reraise(exception.InvalidInput(reason=err_msg)) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/volume/cinder.py", line 226, in _reraise 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2]) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/volume/cinder.py", line 168, in wrapper 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging res = method(self, ctx, *args, **kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/volume/cinder.py", line 190, in wrapper 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging res = method(self, ctx, volume_id, *args, **kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/opt/stack/new/nova/nova/volume/cinder.py", line 315, in begin_detaching 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging cinderclient(context).volumes.begin_detaching(volume_id) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/usr/local/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line 387, in begin_detaching 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return self._action('os-begin_detaching', volume) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/usr/local/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line 334, in _action 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging resp, body = self.api.client.post(url, body=body) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 167, in post 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return self._cs_request(url, 'POST', **kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 155, in _cs_request 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging return self.request(url, method, **kwargs) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 144, in request 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging raise exceptions.from_response(resp, body) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-328765b6-d2ce-4193-a379-dd733d0196bf) 2017-01-31 01:25:10.115 25247 ERROR nova.cells.messaging
Here is the pretty obvious failure in the cells v1 logs:
http:// logs.openstack. org/06/ 425806/ 8/check/ gate-tempest- dsvm-cells- ubuntu- xenial/ ab0bba2/ logs/screen- n-cell- child.txt. gz?level= TRACE#_ 2017-01- 31_01_25_ 10_115
2017-01-31 01:25:10.115 25247 ERROR nova.cells. messaging [req-a820f01f- ff4d-4c62- 8a71-ba509d2061 aa tempest- TestVolumeSwap- 893937839 tempest- TestVolumeSwap- 893937839] Error processing message locally messaging Traceback (most recent call last): messaging File "/opt/stack/ new/nova/ nova/cells/ messaging. py", line 188, in _process_locally messaging resp_value = self.msg_ runner. _process_ message_ locally( self) messaging File "/opt/stack/ new/nova/ nova/cells/ messaging. py", line 1275, in _process_ message_ locally messaging return fn(message, **message. method_ kwargs) messaging File "/opt/stack/ new/nova/ nova/cells/ messaging. py", line 677, in run_compute_ api_method messaging return fn(message.ctxt, *args, **method_ info['method_ kwargs' ]) messaging File "/opt/stack/ new/nova/ nova/compute/ api.py" , line 170, in inner messaging return function(self, context, instance, *args, **kwargs) messaging File "/opt/stack/ new/nova/ nova/compute/ api.py" , line 151, in inner messaging return f(self, context, instance, *args, **kw) messaging File "/opt/stack/ new/nova/ nova/compute/ api.py" , line 3697, in detach_volume messaging self._detach_ volume( context, instance, volume) messaging File "/opt/stack/ new/nova/ nova/compute/ api.py" , line 3665, in _detach_volume messaging self._check_ and_begin_ detach( context, volume, instance) messaging File "/opt/stack/ new/nova/ nova/compute/ api.py" , line 3657, in _check_ and_begin_ detach messaging self.volume_ api.begin_ detaching( context, volume['id']) messaging File "/opt/stack/ new/nova/ nova/volume/ cinder. py", line 176, in wrapper messaging _reraise( exception. InvalidInput( reason= err_msg) ) messaging File "/opt/stack/ new/nova/ nova/volume/ cinder. py", line 226, in _reraise messaging six.reraise( type(desired_ exc), desired_exc, sys.exc_info()[2]) messaging File "/opt/stack/ new/nova/ nova/volume/ cinder. py", line 168, in wrapper messaging res = method(self, ctx, *args, **kwargs) messaging File "/opt/stack/ new/nova/ nova/volume/ cinder. py", line 190, in wrapper messaging res = method(self, ctx, volume_id, *args, **kwargs) messaging File "/opt/stack/ new/nova/ nova/volume/ cinder. py", line 315, in begin_detaching messaging cinderclient( context) .volumes. begin_detaching (volume_ id) messaging File "/usr/local/ lib/python2. 7/dist- packages/ cinderclient/ v2/volumes. py", line 387, in begin_detaching messaging return self._action( 'os-begin_ detaching' , volume) messaging File "/usr/local/ lib/python2. 7/dist- packages/ cinderclient/ v2/volumes. py", line 334, in _action messaging resp, body = self.api. client. post(url, body=body) messaging File "/usr/local/ lib/python2. 7/dist- packages/ cinderclient/ client. py", line 167, in post messaging return self._cs_ request( url, 'POST', **kwargs) messaging File "/usr/local/ lib/python2. 7/dist- packages/ cinderclient/ client. py", line 155, in _cs_request messaging return self.request(url, method, **kwargs) messaging File "/usr/local/ lib/python2. 7/dist- packages/ cinderclient/ client. py", line 144, in request messaging raise exceptions. from_response( resp, body) messaging InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-328765b6- d2ce-4193- a379-dd733d0196 bf) messaging
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.
2017-01-31 01:25:10.115 25247 ERROR nova.cells.