Gate scripts need to remove the awk output trim

Bug #1432295 reported by Kevin Carter
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Critical
Kevin Carter
Icehouse
Won't Fix
Critical
Kevin Carter
Juno
Invalid
Critical
Kevin Carter
Trunk
Fix Released
Critical
Kevin Carter

Bug Description

The Awk output trim found in all of the gate scripts needs to be removed. This output trimming makes it impossible to troubleshoot issues within the playbooks while also having the side effect of not trapping anything that fails. Awk, when used with a pipe, will always return 0 no matter what was done using the command. This means that if a playbook execution were to fail when called bu the gate-check-commit.sh script the gate process would continue until it later failed within tempest or something else.

Awk test case:

``` bash
# Test with executed command
~$ echo hi | awk BOKEN
~$ echo $?
0

# Test with known failed command
~$ false | awk BOKEN
~$ echo $?
0

```

While the filter had good intentions we need to figure out a different solution, however while we are figuring that different solution out we need to remove the filter for accurate gating.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-ansible-deployment (icehouse)

Fix proposed to branch: icehouse
Review: https://review.openstack.org/164481

Revision history for this message
Kevin Carter (kevin-carter) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (master)

Reviewed: https://review.openstack.org/164480
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=b76d43e8c60bcb3060ee4ee99b6fac8acf0650fb
Submitter: Jenkins
Branch: master

commit b76d43e8c60bcb3060ee4ee99b6fac8acf0650fb
Author: Hugh Saunders <email address hidden>
Date: Mon Mar 16 15:26:32 2015 +0000

    Ensure return code passes through output trimming

    Currently the output of run-playbooks is filtered through an awk
    statement to reduce the verbosity of the output. The problem is that
    bash uses the return code from the right most command to determine the
    return code for the whole pipe. Awk successfully processes the
    run-playbooks output, even if run-playbooks fails, so awk returns 0, and
    the pipeline is assumed to have completed successfully.

    From the bash manual:

    "Each command in a pipeline is executed in its own subshell (see Command
    Execution Environment). The exit status of a pipeline is the exit status
    of the last command in the pipeline, unless the pipefail option is
    enabled (see The Set Builtin). If pipefail is enabled, the pipeline’s
    return status is the value of the last (rightmost) command to exit with
    a non-zero status, or zero if all commands exit successfully. If the
    reserved word ‘!’ precedes the pipeline, the exit status is the logical
    negation of the exit status as described above. The shell waits for all
    commands in the pipeline to terminate before returning a value."

    By enabling pipefail, we can ensure that the pipeline exits non-zero if
    the run-playbooks script fails.

    Closes-bug: #1432295
    Change-Id: I019141a7bfbcfa817b90e39f4ae1a68eb3f269a5

Changed in openstack-ansible:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to os-ansible-deployment (icehouse)

Fix proposed to branch: icehouse
Review: https://review.openstack.org/165085

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on os-ansible-deployment (icehouse)

Change abandoned by Jesse Pretorius (<email address hidden>) on branch: icehouse
Review: https://review.openstack.org/164481
Reason: Abandoning in favor of https://review.openstack.org/165085

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (juno)

Reviewed: https://review.openstack.org/163809
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=67be3d09aa043df4411aced6f55131c65a361f0c
Submitter: Jenkins
Branch: juno

commit 67be3d09aa043df4411aced6f55131c65a361f0c
Author: Jesse Pretorius <email address hidden>
Date: Thu Mar 12 11:26:07 2015 +0000

    Adjust method for trimming ansible output debug information

    This patch adjusts the method introduced in
    I98fb57a56c1b7ddd6558e265cf6e65b159a05123 to trim the debug information
    from the ansible output.

    This adjustment allows the output to come to the console immediately,
    instead of piping to a file and doing the processing after the ansible
    run is complete. It also ensures that the return code passes through the
    output trimming.

    This commit combines the following cherry-picks:
    - ed090eb1094472dcef755e6ca319310e5a0e7eac
    - b76d43e8c60bcb3060ee4ee99b6fac8acf0650fb

    Partial-bug: #1425482
    Closes-bug: #1432295
    Co-Authored-By: Hugh Saunders <email address hidden>
    Change-Id: I515adcad34ef76b8ea47fe90bc6572fb3aec1d2c

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to os-ansible-deployment (icehouse)

Reviewed: https://review.openstack.org/165085
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=5ea5eb3d5b7ddf13b1f7265e51a49d168b72b397
Submitter: Jenkins
Branch: icehouse

commit 5ea5eb3d5b7ddf13b1f7265e51a49d168b72b397
Author: Hugh Saunders <email address hidden>
Date: Mon Mar 16 15:26:32 2015 +0000

    Ensure return code passes through output trimming

    Currently the output of run-playbooks is filtered through an awk
    statement to reduce the verbosity of the output. The problem is that
    bash uses the return code from the right most command to determine the
    return code for the whole pipe. Awk successfully processes the
    run-playbooks output, even if run-playbooks fails, so awk returns 0, and
    the pipeline is assumed to have completed successfully.

    From the bash manual:

    "Each command in a pipeline is executed in its own subshell (see Command
    Execution Environment). The exit status of a pipeline is the exit status
    of the last command in the pipeline, unless the pipefail option is
    enabled (see The Set Builtin). If pipefail is enabled, the pipeline’s
    return status is the value of the last (rightmost) command to exit with
    a non-zero status, or zero if all commands exit successfully. If the
    reserved word ‘!’ precedes the pipeline, the exit status is the logical
    negation of the exit status as described above. The shell waits for all
    commands in the pipeline to terminate before returning a value."

    By enabling pipefail, we can ensure that the pipeline exits non-zero if
    the run-playbooks script fails.

    Closes-bug: #1432295
    Change-Id: I019141a7bfbcfa817b90e39f4ae1a68eb3f269a5
    (cherry picked from commit b76d43e8c60bcb3060ee4ee99b6fac8acf0650fb)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.