The issue that I'm facing relates to the fact that the output has changed, meaning that if the 'Code' is different from 0 (successful) it doesn't return a dictionary with the key 'Stdout', which leads it to fail because it is looking for something that doesn't exist. The output is the following in UOSCI: 2019-10-04 14:34:27 [INFO] ERROR: test_901_pause_resume (zaza.openstack.charm_tests.ceilometer.tests.CeilometerTest) 2019-10-04 14:34:27 [INFO] Run pause and resume tests. 2019-10-04 14:34:27 [INFO] ---------------------------------------------------------------------- 2019-10-04 14:34:27 [INFO] Traceback (most recent call last): 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/openstack/charm_tests/ceilometer/tests.py", line 121, in test_901_pause_resume 2019-10-04 14:34:27 [INFO] with self.pause_resume(self.services): 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__ 2019-10-04 14:34:27 [INFO] return next(self.gen) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/openstack/charm_tests/test_utils.py", line 283, in pause_resume 2019-10-04 14:34:27 [INFO] pgrep_full=pgrep_full) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/__init__.py", line 48, in _wrapper 2019-10-04 14:34:27 [INFO] return run(_run_it()) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/__init__.py", line 36, in run 2019-10-04 14:34:27 [INFO] return task.result() 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result 2019-10-04 14:34:27 [INFO] raise self._exception 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step 2019-10-04 14:34:27 [INFO] result = coro.send(None) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/__init__.py", line 47, in _run_it 2019-10-04 14:34:27 [INFO] return await f(*args, **kwargs) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/model.py", line 988, in async_block_until_service_status 2019-10-04 14:34:27 [INFO] await async_block_until(_check_service, timeout=timeout) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/model.py", line 1056, in async_block_until 2019-10-04 14:34:27 [INFO] await asyncio.wait_for(_block(), timeout, loop=loop) 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/asyncio/tasks.py", line 392, in wait_for 2019-10-04 14:34:27 [INFO] return fut.result() 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result 2019-10-04 14:34:27 [INFO] raise self._exception 2019-10-04 14:34:27 [INFO] File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step 2019-10-04 14:34:27 [INFO] result = coro.send(None) 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/model.py", line 1050, in _block 2019-10-04 14:34:27 [INFO] result = await c() 2019-10-04 14:34:27 [INFO] File "/tmp/tmp.okQAPDUoyT/func-smoke/lib/python3.5/site-packages/zaza/model.py", line 981, in _check_service 2019-10-04 14:34:27 [INFO] response_size = len(out['Stdout'].strip()) 2019-10-04 14:34:27 [INFO] KeyError: 'Stdout' 2019-10-04 14:34:27 [INFO] ---------------------------------------------------------------------- 2019-10-04 14:34:27 [INFO] Ran 2 tests in 186.632s 2019-10-04 14:34:27 [INFO] FAILED 2019-10-04 14:34:27 [INFO] (errors=1) I've also proposed the following patch to Zaza, in order to solve this problem: https://github.com/openstack-charmers/zaza/pull/301 It is a fix but further development should be done in order to account for other possibilities that might arise from other possible values.