[master] scenario000 multinode upgrade job is failing at tripleo-upgrade include task giving controllers' is undefined"}

[chandan kumar]

Master scenario000 multinode upgrade jobs are failing at [tripleo-upgrade : include] task while Run tripleo-upgrade role to upgrade overcloud with following error:

2018-07-24 07:07:01.221500 | primary |
2018-07-24 07:07:01.221711 | primary | PLAY [Run tripleo-upgrade role to upgrade overcloud] ***************************
2018-07-24 07:07:01.616314 | primary |
2018-07-24 07:07:01.616535 | primary | TASK [tripleo-upgrade : include] ***********************************************
2018-07-24 07:07:01.651367 | primary | Tuesday 24 July 2018 07:07:01 +0000 (0:00:08.592) 0:00:34.246 **********
2018-07-24 07:07:01.701729 | primary | fatal: [undercloud]: FAILED! => {"msg": "'controllers' is undefined"}

currently two patches which are independent are hitting with this error
http://logs.openstack.org/61/584761/1/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/745f895/job-output.txt.gz#_2018-07-24_07_38_33_170569

http://logs.openstack.org/14/584314/6/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/41f570c/job-output.txt.gz#_2018-07-24_07_07_01_701729

I am not sure but something is wrong going here https://github.com/openstack/tripleo-upgrade/blob/4572055ffb52b9fe6ff3567afe709a0a68cf1d6b/tasks/common/controller_post_scripts.yml#L5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/585654

[Jose Luis Franco (jfrancoa)]

Besides the submitted patch, there is also an issue with the tags used in the job. This is an upgrades job, but there are not "overcloud-upgrade" tag passed in the --tags argument:

--tags build,undercloud-setup,undercloud-scripts,undercloud-install,undercloud-post-install,tripleo-validations,overcloud-scripts,overcloud-prep-config,overcloud-prep-containers,overcloud-deploy,overcloud-post-deploy,overcloud-validate

[Bogdan Dobrelya (bogdando)]

Then we prolly can make https://review.openstack.org/#/c/585528/ fixing this bug as well! Let's add missing tags?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/585654
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=629309217b13118ca83016cd9553c8a426c8a31d
Submitter: Zuul
Branch: master

commit 629309217b13118ca83016cd9553c8a426c8a31d
Author: Jose Luis Franco Arza <email address hidden>
Date: Wed Jul 25 08:48:46 2018 +0200

    Change include for include_tasks when including loop controled scripts.

    We obtain a 'controllers' is undefined ansible error when
    the tripleo-upgrade is run without any role related tag
    (e.g. overcloud_upgrade, overcloud_update, etc...). The
    issue seems to appear at the time of including the
    controller_post_script.yml tasks, which content is defined
    depending on the result of a shell command. As the command
    is not executed during the role importing, the include fails.

    The new include_tasks keyword seems to handle this issue
    without displaying any error. Also the include keyword was
    deprecated in Ansible 2.4, so it's time to start moving into
    include_tasks/import_tasks.

    Change-Id: I9d10e1d9a95e11fd59bf593bb052abf9eb17cf78
    Closes-Bug: #1783303

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/593595

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/593595
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=7357d5714fa9b9a9aafb118347c91fdb11c2eef7
Submitter: Zuul
Branch: stable/queens

commit 7357d5714fa9b9a9aafb118347c91fdb11c2eef7
Author: Jose Luis Franco Arza <email address hidden>
Date: Wed Jul 25 08:48:46 2018 +0200

    Change include for include_tasks when including loop controled scripts.

    We obtain a 'controllers' is undefined ansible error when
    the tripleo-upgrade is run without any role related tag
    (e.g. overcloud_upgrade, overcloud_update, etc...). The
    issue seems to appear at the time of including the
    controller_post_script.yml tasks, which content is defined
    depending on the result of a shell command. As the command
    is not executed during the role importing, the include fails.

    The new include_tasks keyword seems to handle this issue
    without displaying any error. Also the include keyword was
    deprecated in Ansible 2.4, so it's time to start moving into
    include_tasks/import_tasks.

    Change-Id: I9d10e1d9a95e11fd59bf593bb052abf9eb17cf78
    Closes-Bug: #1783303
    (cherry picked from commit 629309217b13118ca83016cd9553c8a426c8a31d)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-upgrade 3.0.0 release.