Looks like create failed. We have a few exceptions:
http://logs.openstack.org/76/461076/2/check/gate-tripleo-ci-centos-7-scenario002-multinode-upgrades/bdd7f5b/logs/subnode-2/var/log/cinder/volume.txt.gz#_2017-06-02_23_37_29_880
2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume [req-0f53120c-8a61-4066-bbef-732ef3cca75d 6757101f42a948a9818786ceb8528430 ad580cee7399423db9d0eaa3acf0a63f - default default] Failed to copy image e5a605c5-0a8b-47e7-8b55-93793ed5b84a to volume: 89d6a97e-5541-435f-9e03-3b58c57c4955 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume Traceback (most recent call last): 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 535, in _copy_image_to_volume 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 809, in copy_image_to_encrypted_volume 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume context, volume, image_service, image_id, True) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 827, in _copy_image_data_to_volume 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume encryption) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 514, in brick_attach_volume_encryptor 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume encryptor.attach_volume(context, **encryption) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/encryptors/luks.py", line 150, in attach_volume 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume key = self._get_key(context).get_encoded() 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/os_brick/encryptors/base.py", line 50, in _get_key 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume return self._key_manager.get(context, self.encryption_key_id) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume File "/usr/lib/python2.7/site-packages/cinder/keymgr/conf_key_mgr.py", line 128, in get 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume raise KeyError(str(managed_object_id) + " != " + str(self.key_id)) 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume KeyError: 'e3c00430-5907-4795-8719-7f2bca191b86 != 00000000-0000-0000-0000-000000000000' 2017-06-02 23:37:29.880 210553 ERROR cinder.volume.flows.manager.create_volume
http://logs.openstack.org/76/461076/2/check/gate-tripleo-ci-centos-7-scenario002-multinode-upgrades/bdd7f5b/logs/subnode-2/var/log/cinder/volume.txt.gz#_2017-06-02_23_37_29_919
2017-06-02 23:37:29.919 210553 ERROR cinder.volume.flows.manager.create_volume [req-0f53120c-8a61-4066-bbef-732ef3cca75d 6757101f42a948a9818786ceb8528430 ad580cee7399423db9d0eaa3acf0a63f - default default] Volume 89d6a97e-5541-435f-9e03-3b58c57c4955: create failed
http://logs.openstack.org/76/461076/2/check/gate-tripleo-ci-centos-7-scenario002-multinode-upgrades/bdd7f5b/logs/subnode-2/var/log/cinder/volume.txt.gz#_2017-06-02_23_37_29_938
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server [req-0f53120c-8a61-4066-bbef-732ef3cca75d 6757101f42a948a9818786ceb8528430 ad580cee7399423db9d0eaa3acf0a63f - default default] Exception during message handling 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "<string>", line 2, in create_volume 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 654, in create_volume 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server _run_flow() 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 646, in _run_flow 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server flow_engine.run() 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout): 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server failures[0].reraise() 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server result = task.execute(**arguments) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 902, in execute 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server **volume_spec) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 849, in _create_from_image 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server image_service) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "<string>", line 2, in _create_from_image_cache_or_download 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/coordination.py", line 301, in _synchronized 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server return f(*a, **k) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 768, in _create_from_image_cache_or_download 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server image_service 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 675, in _create_from_image_download 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server image_service) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 553, in _copy_image_to_volume 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex) 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server ImageCopyFailure: Failed to copy image to volume: 'e3c00430-5907-4795-8719-7f2bca191b86 != 00000000-0000-0000-0000-000000000000' 2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.rpc.server
Looks like create failed. We have a few exceptions:
http:// logs.openstack. org/76/ 461076/ 2/check/ gate-tripleo- ci-centos- 7-scenario002- multinode- upgrades/ bdd7f5b/ logs/subnode- 2/var/log/ cinder/ volume. txt.gz# _2017-06- 02_23_37_ 29_880
2017-06-02 23:37:29.880 210553 ERROR cinder. volume. flows.manager. create_ volume [req-0f53120c- 8a61-4066- bbef-732ef3cca7 5d 6757101f42a948a 9818786ceb85284 30 ad580cee7399423 db9d0eaa3acf0a6 3f - default default] Failed to copy image e5a605c5- 0a8b-47e7- 8b55-93793ed5b8 4a to volume: 89d6a97e- 5541-435f- 9e03-3b58c57c49 55 volume. flows.manager. create_ volume Traceback (most recent call last): volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 535, in _copy_image_ to_volume volume. flows.manager. create_ volume context, volume, image_service, image_id) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ driver. py", line 809, in copy_image_ to_encrypted_ volume volume. flows.manager. create_ volume context, volume, image_service, image_id, True) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ driver. py", line 827, in _copy_image_ data_to_ volume volume. flows.manager. create_ volume encryption) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ cinder/ utils.py" , line 514, in brick_attach_ volume_ encryptor volume. flows.manager. create_ volume encryptor. attach_ volume( context, **encryption) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ os_brick/ encryptors/ luks.py" , line 150, in attach_volume volume. flows.manager. create_ volume key = self._get_ key(context) .get_encoded( ) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ os_brick/ encryptors/ base.py" , line 50, in _get_key volume. flows.manager. create_ volume return self._key_ manager. get(context, self.encryption _key_id) volume. flows.manager. create_ volume File "/usr/lib/ python2. 7/site- packages/ cinder/ keymgr/ conf_key_ mgr.py" , line 128, in get volume. flows.manager. create_ volume raise KeyError( str(managed_ object_ id) + " != " + str(self.key_id)) volume. flows.manager. create_ volume KeyError: 'e3c00430- 5907-4795- 8719-7f2bca191b 86 != 00000000- 0000-0000- 0000-0000000000 00' volume. flows.manager. create_ volume
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
2017-06-02 23:37:29.880 210553 ERROR cinder.
http:// logs.openstack. org/76/ 461076/ 2/check/ gate-tripleo- ci-centos- 7-scenario002- multinode- upgrades/ bdd7f5b/ logs/subnode- 2/var/log/ cinder/ volume. txt.gz# _2017-06- 02_23_37_ 29_919
2017-06-02 23:37:29.919 210553 ERROR cinder. volume. flows.manager. create_ volume [req-0f53120c- 8a61-4066- bbef-732ef3cca7 5d 6757101f42a948a 9818786ceb85284 30 ad580cee7399423 db9d0eaa3acf0a6 3f - default default] Volume 89d6a97e- 5541-435f- 9e03-3b58c57c49 55: create failed
http:// logs.openstack. org/76/ 461076/ 2/check/ gate-tripleo- ci-centos- 7-scenario002- multinode- upgrades/ bdd7f5b/ logs/subnode- 2/var/log/ cinder/ volume. txt.gz# _2017-06- 02_23_37_ 29_938
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging. rpc.server [req-0f53120c- 8a61-4066- bbef-732ef3cca7 5d 6757101f42a948a 9818786ceb85284 30 ad580cee7399423 db9d0eaa3acf0a6 3f - default default] Exception during message handling rpc.server Traceback (most recent call last): rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/server. py", line 155, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 222, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/usr/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 192, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "<string>", line 2, in create_volume rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ objects/ cleanable. py", line 207, in wrapper rpc.server result = f(*args, **kwargs) rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ manager. py", line 654, in create_volume rpc.server _run_flow() rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ manager. py", line 646, in _run_flow rpc.server flow_engine.run() rpc.server File "/usr/lib/ python2. 7/site- packages/ taskflow/ engines/ action_ engine/ engine. py", line 247, in run rpc.server for _state in self.run_ iter(timeout= timeout) : rpc.server File "/usr/lib/ python2. 7/site- packages/ taskflow/ engines/ action_ engine/ engine. py", line 340, in run_iter rpc.server failure. Failure. reraise_ if_any( er_failures) rpc.server File "/usr/lib/ python2. 7/site- packages/ taskflow/ types/failure. py", line 336, in reraise_if_any rpc.server failures[ 0].reraise( ) rpc.server File "/usr/lib/ python2. 7/site- packages/ taskflow/ types/failure. py", line 343, in reraise rpc.server six.reraise( *self._ exc_info) rpc.server File "/usr/lib/ python2. 7/site- packages/ taskflow/ engines/ action_ engine/ executor. py", line 53, in _execute_task rpc.server result = task.execute( **arguments) rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 902, in execute rpc.server **volume_spec) rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 849, in _create_from_image rpc.server image_service) rpc.server File "<string>", line 2, in _create_ from_image_ cache_or_ download rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ coordination. py", line 301, in _synchronized rpc.server return f(*a, **k) rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 768, in _create_ from_image_ cache_or_ download rpc.server image_service rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 675, in _create_ from_image_ download rpc.server image_service) rpc.server File "/usr/lib/ python2. 7/site- packages/ cinder/ volume/ flows/manager/ create_ volume. py", line 553, in _copy_image_ to_volume rpc.server raise exception. ImageCopyFailur e(reason= ex) rpc.server ImageCopyFailure: Failed to copy image to volume: 'e3c00430- 5907-4795- 8719-7f2bca191b 86 != 00000000- 0000-0000- 0000-0000000000 00' rpc.server
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.
2017-06-02 23:37:29.938 210553 ERROR oslo_messaging.