Functional test intermittent failure

[Liam Young]

There seems tro be an intermitant failure with the functional tests around pause/resume. In this case it was with bionic/queens:

020-06-03 12:11:56 [INFO] Changing settings on designate to {'debug': 'True'}
2020-06-03 12:12:00 [INFO] Waiting for at least one unit with agent status "executing"
2020-06-03 12:13:00 [INFO] ERROR
2020-06-03 12:13:00 [INFO] test_910_pause_and_resume (zaza.openstack.charm_tests.designate.tests.DesignateTests)
2020-06-03 12:13:00 [INFO] Run pause and resume tests.
2020-06-03 12:13:00 [INFO] ...
2020-06-03 12:15:35 [INFO] Testing pause resume
2020-06-03 12:16:03 [INFO] ok
2020-06-03 12:16:03 [INFO] ======================================================================
2020-06-03 12:16:03 [INFO] ERROR: test_900_restart_on_config_change (zaza.openstack.charm_tests.designate.tests.DesignateTests)
2020-06-03 12:16:03 [INFO] Checking restart happens on config change.
2020-06-03 12:16:03 [INFO] ----------------------------------------------------------------------
2020-06-03 12:16:03 [INFO] Traceback (most recent call last):
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/openstack/charm_tests/designate/tests.py", line 239, in test_900_restart_on_config_change
2020-06-03 12:16:03 [INFO] self.designate_svcs,
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/openstack/charm_tests/test_utils.py", line 293, in restart_on_changed_debug_oslo_config_file
2020-06-03 12:16:03 [INFO] services)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/openstack/charm_tests/test_utils.py", line 330, in restart_on_changed
2020-06-03 12:16:03 [INFO] with self.config_change(default_config, alternate_config):
2020-06-03 12:16:03 [INFO] File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
2020-06-03 12:16:03 [INFO] return next(self.gen)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/openstack/charm_tests/test_utils.py", line 231, in config_change
2020-06-03 12:16:03 [INFO] model.wait_for_agent_status(model_name=self.model_name)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/__init__.py", line 48, in _wrapper
2020-06-03 12:16:03 [INFO] return run(_run_it())
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/__init__.py", line 36, in run
2020-06-03 12:16:03 [INFO] return task.result()
2020-06-03 12:16:03 [INFO] File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
2020-06-03 12:16:03 [INFO] raise self._exception
2020-06-03 12:16:03 [INFO] File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
2020-06-03 12:16:03 [INFO] result = coro.send(None)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/__init__.py", line 47, in _run_it
2020-06-03 12:16:03 [INFO] return await f(*args, **kwargs)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/model.py", line 914, in async_wait_for_agent_status
2020-06-03 12:16:03 [INFO] lambda: one_agent_status(model, status), timeout=timeout)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/juju/model.py", line 718, in block_until
2020-06-03 12:16:03 [INFO] loop=self.loop)
2020-06-03 12:16:03 [INFO] File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/juju/utils.py", line 85, in block_until
2020-06-03 12:16:03 [INFO] await asyncio.wait_for(_block(), timeout, loop=loop)
2020-06-03 12:16:03 [INFO] File "/usr/lib/python3.5/asyncio/tasks.py", line 396, in wait_for
2020-06-03 12:16:03 [INFO] raise futures.TimeoutError()
2020-06-03 12:16:03 [INFO] concurrent.futures._base.TimeoutError
2020-06-03 12:16:03 [INFO] ----------------------------------------------------------------------
2020-06-03 12:16:03 [INFO] Ran 4 tests in 275.463s
2020-06-03 12:16:03 [INFO] FAILED
2020-06-03 12:16:03 [INFO] (errors=1)
Traceback (most recent call last):
  File "/tmp/tmp.s5T8D2efSK/func/bin/functest-run-suite", line 8, in <module>
    sys.exit(main())
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 204, in main
    force=args.force)
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 141, in func_test_runner
    force=force)
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 88, in run_env_deployment
    test_steps.get(deployment.model_alias, []))
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/test.py", line 117, in test
    run_test_list(tests)
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/test.py", line 111, in run_test_list
    get_test_runners()[runner](testcase, _testcase)
  File "/tmp/tmp.s5T8D2efSK/func/lib/python3.5/site-packages/zaza/charm_lifecycle/test.py", line 73, in run_unittest
    assert test_result.wasSuccessful(), "Test run failed"
AssertionError: Test run failed

[Alex Kavanagh (ajkavanagh)]

I think this has been overtaken by events as the zosci reports that the designate charm testing is stable now.