tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates is timing out

[wes hayutin]

http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates

http://logs.openstack.org/70/585370/2/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/027eb5f/job-output.txt.gz#_2018-08-15_11_29_33_335200

2018-08-15 10:09:28.139716 | primary | TASK [tripleo-upgrade : setup HEAT outputs via update prepare] *****************
2018-08-15 10:09:28.209052 | primary | Wednesday 15 August 2018 10:09:28 +0000 (0:00:01.001) 0:01:29.616 ******
2018-08-15 11:29:33.335200 | RUN END RESULT_TIMED_OUT: [untrusted : git.openstack.org/openstack-infra/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master]
2018-08-15 11:29:33.336108 | POST-RUN START: [untrusted : git.openstack.org/openstack-infra/tripleo-ci/playbooks/tripleo-ci/post.yaml@master]
2018-08-15 11:29:36.257260 |

[Marios Andreou (marios-b)]

o/ Wes, adding comment as I spent some time trying to understand (rover). Looking at the list [0] of zuul build the job isn't in too bad shape. There are a couple of things going on that I can see at least as follows - one is related to removing the container-registry param [1,2] and it should be solved once we get a new enough tripleo-common more below, and the other is related to nova-cert and so possibly related to the other new bug @ [3]

  * for the container registry parameter removal [1,2] it could only happen if you are using a python-tripleoclient with the change, and then tripleo-common w/out it. There is depends on though and they both merged 3 days ago. The multinode-oooq-container-updates job is green there too. However here you can see the error in [4,5] and it looks like

  2018-08-15 10:11:40.190 ERROR /var/log/containers/mistral/engine.log: 7 ERROR mistral.engine.task_handler [req-fd8fbc0e-350e-4297-8a63-6eb0060ff25a 2bea853d47d643b38acfbd8f7c91504b 7f74b99fd5cc4cf0abaefee63f693317 - default default] Failed to run task [error=Invalid input [name=tripleo.package_update.update_stack, class=tripleo_common.actions.package_update.UpdateStackAction, missing=['container_registry']], wf=tripleo.package_update.v1.package_update_plan, task=update]:

  * for the nova cert error examples are at [5] (immediately following that container registry issue) and also [6] and looks like

  2018-08-15 09:32:20.998 ERROR /var/log/containers/mistral/mistral-db-manage.log: 11 ERROR mistral.actions.openstack.action_generator.base [-] Failed to create action: nova.certs_convert_into_with_meta: AttributeError: 'Client' object has no attribute 'certs'

[0] http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates
[1] https://review.openstack.org/#/c/570893/ python-tripleoclient
[2] https://review.openstack.org/#/c/571186/ tripleo-common
[3] https://bugs.launchpad.net/tripleo/+bug/1787227
[4] http://logs.openstack.org/48/588148/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/8a5b8b2/logs/undercloud/var/log/extra/errors.txt.gz#_2018-08-16_08_22_08_249
[5] http://logs.openstack.org/70/585370/2/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/027eb5f/logs/undercloud/var/log/extra/errors.txt.gz#_2018-08-15_09_32_20_998
[6] http://logs.openstack.org/48/588148/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/8a5b8b2/logs/undercloud/var/log/extra/errors.txt.gz#_2018-08-16_07_33_58_362