looking at heat-engine logs on the undercloud: 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension [-] Could not load 'mistral': No module named mistralclient.api 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension [-] No module named mistralclient.api 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension Traceback (most recent call last): 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension plugin = ep.resolve() 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2235, in resolve 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0) 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/mistral.py", line 14, in 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension from mistralclient.api import base as mistral_base 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension ImportError: No module named mistralclient.api 2016-08-01 14:20:37.315 16992 ERROR stevedore.extension 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension [-] Could not load 'magnum': No module named magnumclient.openstack.common.apiclient 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension [-] No module named magnumclient.openstack.common.apiclient 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension Traceback (most recent call last): 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension plugin = ep.resolve() 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2235, in resolve 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0) 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/magnum.py", line 14, in 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension from magnumclient.openstack.common.apiclient import exceptions 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension ImportError: No module named magnumclient.openstack.common.apiclient 2016-08-01 14:20:37.340 16992 ERROR stevedore.extension 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension [-] Could not load 'barbican': No module named barbicanclient 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension [-] No module named barbicanclient 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension Traceback (most recent call last): 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension plugin = ep.resolve() 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2235, in resolve 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0) 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/barbican.py", line 18, in 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension from barbicanclient import client as barbican_client 2016-08-01 14:20:37.365 16992 ERROR stevedore.extension ImportError: No module named barbicanclient 6-08-01 14:20:37.365 16992 ERROR stevedore.extension 2016-08-01 14:20:37.391 16992 ERROR stevedore.extension [-] Could not load 'designate': No module named designateclient 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension [-] No module named designateclient 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension Traceback (most recent call last): 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/enabled.py", line 67, in _load_one_plugin 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension plugin = ep.resolve() 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2235, in resolve 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0) 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/designate.py", line 14, in 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension from designateclient import exceptions 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension ImportError: No module named designateclient 2016-08-01 14:20:37.392 16992 ERROR stevedore.extension 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension [-] Could not load 'designate.domain': No module named designateclient 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension [-] No module named designateclient 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension Traceback (most recent call last): 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 162, in _load_plugins 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension verify_requirements, 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 183, in _load_one_plugin 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension plugin = ep.resolve() 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2235, in resolve 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension module = __import__(self.module_name, fromlist=['__name__'], level=0) 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/designate.py", line 14, in 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension from designateclient import exceptions 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension ImportError: No module named designateclient 2016-08-01 14:20:37.394 16992 ERROR stevedore.extension later I see: 2016-08-01 15:52:28.601 13562 DEBUG heat.engine.scheduler [-] Task _run_to_completion from ResourceGroup "Compute" [5723785e-e1e0-400c-99d8-30b575b2e53b] Stack "overcloud" [f69e9614-1b69-4ec 1-8c99-a03c9a9db540] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:216 2016-08-01 15:52:28.606 13562 INFO heat.engine.resource [-] UPDATE: ResourceGroup "Compute" [5723785e-e1e0-400c-99d8-30b575b2e53b] Stack "overcloud" [f69e9614-1b69-4ec1-8c99-a03c9a9db540] 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource Traceback (most recent call last): 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 704, in _action_recorder 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource yield 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1152, in update 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource prop_diff]) 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 314, in wrapper 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource step = next(subtask) 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 749, in action_handler_task 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource while not check(handler_data): 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 382, in check_update_complete 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource if not checker.step(): 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 219, in step 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource next(self._runner) 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 370, in _run_to_completion 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource self).check_update_complete(updater): 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 443, in check_update_complete 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource cookie=cookie) 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 390, in _check_status_complete 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource action=action) 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource ResourceFailure: resources.Compute: Engine went down during stack UPDATE 2016-08-01 15:52:28.606 13562 ERROR heat.engine.resource 2016-08-01 15:52:28.612 13565 DEBUG oslo_messaging._drivers.amqpdriver [req-7895c12a-90b6-4ec2-ab90-fe729ac250f1 admin admin - default default] sending reply msg_id: 3576e370893f46f6b3ef14b871a3a6ae reply queue: reply_e5e5495ac0f346d1b83ba3210acd152f time elapsed: 0.0155863330001s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-01 15:52:28.616 13565 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 3576e370893f46f6b3ef14b871a3a6ae __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:302