scenario001/2-multinode-oooq-container deployment fails and timesout, containers not deployed

[wes hayutin]

scenario001 and scenario002 have been running red for about 24 hours [1-2]. They have been timing out during the overcloud deployment so we never have collected all the logs from the deployment.

Slagle pointed out the issue may be found in
http://logs.openstack.org/23/540323/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e8cc91/logs/undercloud/var/log/deployed-server-os-collect-config.log.txt.gz

and caused by:
https://review.openstack.org/#/c/537375/

Testing a revert here:
https://review.openstack.org/#/c/540545/

[1] http://tripleo.org/cistatus.html
[2] http://cistatus.tripleo.org/#tripleo-ci-centos-7-scenario001-multinode-oooq-container
[2] http://cistatus.tripleo.org/#tripleo-ci-centos-7-scenario002-multinode-oooq-container

[wes hayutin (weshayutin)]

http://logs.openstack.org/23/540323/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2e8cc91/
http://logs.openstack.org/45/521545/19/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/77cdaea/
http://logs.openstack.org/19/537719/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/18be002/
http://logs.openstack.org/85/540385/1/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/df4505c/
http://logs.openstack.org/23/540323/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/4fcc6f9/
http://logs.openstack.org/45/521545/19/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container/18acd6c/
http://logs.openstack.org/19/537719/1/gate/tripleo-ci-centos-7-scenario002-multinode-oooq-container/92dbe8b/

[wes hayutin (weshayutin)]

we should update the jobs to catch these issues https://review.openstack.org/#/c/540548/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/540543
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f17b0a14dad89fdb0594457b602d4ca183b230d7
Submitter: Zuul
Branch: master

commit f17b0a14dad89fdb0594457b602d4ca183b230d7
Author: wes hayutin <email address hidden>
Date: Fri Feb 2 22:20:03 2018 +0000

    Revert "Introduce wait_backoff, fail_if_empty wrappers"

    This reverts commit d5be6a272216fd679882fd2bb86114e7871192ce.
    Tested with https://review.openstack.org/#/c/540545/

    Closes-Bug: #1747089
    Change-Id: I96c0a15ae7068c067ff99ee84218dc52159933b9

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.0.0rc1 release candidate.