[Debug output for task: Run puppet host configuration for step 1] TASK is skipping failures

[Ayman Mafarja]

Hi,

The task [Debug output for task: Run puppet host configuration for step 1] skip failures in the ansible playbook tripleo-heat-templates/common/deploy-steps-tasks.yaml.

This causes failures in some other components and blocks controller replacement procedure.

After executing a stack update, I got a failure in the mentioned task but the process skipped it.

LOGS:
TASK [Debug output for task: Run puppet host configuration for step 1] *********

Sunday 14 July 2019 10:41:14 +0300 (0:01:49.119) 0:33:19.230 ***********

fatal: [overcloud-controller-london-3]: FAILED! => {
...
        \"Error: /sbin/pcs status | grep -q 'partition with quorum' > /dev/null 2>&1 returned 1 instead of one of [0]\",

        \"Error: /Stage[main]/Pacemaker::Corosync/Exec[wait-for-settle]/returns: change from notrun to 0 failed: /sbin/pcs status | grep -q 'partition with quorum' > /dev/null 2>&1 returned 1 instead of one of [0]\"

}
ok: [overcloud-controller-london-1] => {
...
}

TASK [Run docker-puppet tasks (generate config) during step 1] *****************
...

Note: this process used to stop upon failures but it is currently skipping failures.

Expected behavior: to stop the process after having a failure
Actual behavior: failues are skipped

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/671093

[Ayman Mafarja (amafarja)]

Unfortunately, the fix made all nodes fail in that task.

Logs:
TASK [Run puppet host configuration for step 1] ********************************
Wednesday 17 July 2019 00:52:32 +0300 (0:00:01.811) 0:19:37.755 ********
fatal: [overcloud-controller-3]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [overcloud-controller-1]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [overcloud-controller-0]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [overcloud-ovscompute-0]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/671093
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=7f2e8b875ae566ce3b6e0c7090d56300b1c55b16
Submitter: Zuul
Branch: master

commit 7f2e8b875ae566ce3b6e0c7090d56300b1c55b16
Author: Bogdan Dobrelya <email address hidden>
Date: Tue Jul 16 17:47:11 2019 +0200

    Use pipefail instead of checking PIPESTATUS

    It seems that the pipefail option works more reliably than
    $PIPESTATUS. The former also provides an established pattern for
    the shell commands executed via ansible in tripleo.

    Closes-bug: #1836596

    Change-Id: Ic70a11281b471a51698490bd899c99dafaba1e35
    Signed-off-by: Bogdan Dobrelya <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.1.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/704801

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/stein
Review: https://review.opendev.org/704801
Reason: It seems that stable/stein is long forgotten, so this is to be forever lost in the ether.