overcloud-prep-containers not executing at featureset016

[Quique Llorente]

Could be the cause of the timeouts, overcloud-prep-containers is being skipped
http://logs.openstack.org/71/601671/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9ce841f/job-output.txt.gz#_2018-09-12_05_13_34_611866
2018-09-12 05:13:34.611866 | primary | TASK [overcloud-prep-containers : Prepare for the containerized deployment] ****
2018-09-12 05:13:34.650570 | primary | Wednesday 12 September 2018 05:13:34 +0000 (0:00:00.114) 0:00:48.555 ***
2018-09-12 05:13:34.677875 | primary | skipping: [undercloud]

[Quique Llorente (quiquell)]

Looking at the moving average, looks like time it started to increase in 31/08

Before http://logs.openstack.org/93/596093/7/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/8c1f92d/logs/ara_oooq_root/

After http://logs.openstack.org/71/601671/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9ce841f/logs/ara_oooq_root/

[Quique Llorente (quiquell)]

We are running an upgrade related play
http://logs.openstack.org/71/601671/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9ce841f/job-output.txt.gz#_2018-09-12_06_21_21_121569
2018-09-12 06:21:21.121569 | primary | PLAY [Generate repo-setup script for overcloud upgrade] ************************
2018-09-12 06:21:21.170404 | primary |

[Quique Llorente (quiquell)]

Playbook list is wrong http://logs.openstack.org/71/601671/3/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9ce841f/job-output.txt.gz#_2018-09-12_06_21_17_325117
+(/home/zuul/src/git.openstack.org/openstack//tripleo-ci/scripts/oooq_common_functions.sh:94): run_/usr/bin/timeout --preserve-status 32m /home/zuul/workspace//.quickstart/bin/ansible-playbook --tags build,undercloud-setup,undercloud-scripts,undercloud-install,undercloud-post-install,tripleo-validations,overcloud-scripts,overcloud-prep-config,overcloud-prep-containers,overcloud-deploy,overcloud-post-deploy,overcloud-validate --skip-tags teardown-all --extra-vars @/home/zuul/workspace//.quickstart/config/release/tripleo-ci/master.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack//tripleo-quickstart/config/nodes/1ctlr.yml --extra-vars @/home/zuul/workspace/.quickstart/config/general_config/featureset-multinode-common.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack//tripleo-quickstart/config/general_config/featureset016.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack//tripleo-ci/toci-quickstart/config/testenv/multinode.yml -e toci_vxlan_networking=false -e vxlan_networking=false --extra-vars local_working_dir=/home/zuul/workspace//.quickstart --extra-vars virthost=127.0.0.2 --inventory /home/zuul/workspace//.quickstart/hosts --extra-vars tripleo_root=/home/zuul/src/git.openstack.org/openstack/ --extra-vars working_dir=/home/zuul --extra-vars ci_job_end_time=1536735367 /home/zuul/workspace//.quickstart/playbooks/multinode-overcloud-upgrade.yml

[Quique Llorente (quiquell)]

issue: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/base.yaml#L66

[chandan kumar (chkumar246)]

More job logs which are getting skipped at this step from 11/09:
* http://logs.openstack.org/24/601324/1/gate/tripleo-ci-centos-7-containers-multinode/7f1ca3c/job-output.txt.gz#_2018-09-11_07_36_36_425525

* http://logs.openstack.org/31/591731/3/gate/tripleo-ci-centos-7-scenario004-multinode-oooq-container/95bc800/job-output.txt.gz#_2018-09-11_00_37_59_785784

* http://logs.openstack.org/31/591731/3/gate/tripleo-ci-centos-7-scenario003-multinode-oooq-container/4701c03/job-output.txt.gz#_2018-09-11_00_52_37_084012

* http://logs.openstack.org/74/563874/7/gate/tripleo-ci-centos-7-scenario007-multinode-oooq-container/dbd2d08/job-output.txt.gz#_2018-09-10_21_49_55_777193

Timed out job where this step is passed:
http://logs.openstack.org/74/601074/1/gate/tripleo-ci-centos-7-scenario002-multinode-oooq-container/f43101a/job-output.txt.gz#_2018-09-11_04_03_39_332524

[Quique Llorente (quiquell)]

overcloud-upgrade tag is nos selected, the upgrade playbook should not run.

[Quique Llorente (quiquell)]

Upgrades was a red herring

[Quique Llorente (quiquell)]

To print "containerized_overcloud" https://review.openstack.org/601956

[Quique Llorente (quiquell)]

Looks like we are not suppose to run the skipped stuff https://review.openstack.org/#/c/573476/