Task conditionals are not consistently parsed when doing tasks file.

[Sofer Athlan-Guyot]

Hi,

When splitting the tasks by file we parse the conditional step to put the tasks in the right step file.

This process may fail:

2020-07-10 01:57:36 | 2020-07-10 01:57:36.562501 | bc764e10-263d-604f-48bc-000000000031 | TIMING | Enforce RHOSP rules regarding subscription. | 0:00:31.769 | 0.14s

2020-07-10 01:57:36 | 2020-07-10 01:57:36.625485 | bc764e10-263d-604f-48bc-000000000032 | FATAL | Exit if existing yum process | centos-8-rax-ord-0018017813 | error={

2020-07-10 01:57:36 | "msg": "The conditional check '(step|int == 0 or step|int == 3) and yum_pid_file.stat.exists' failed. The error was: error while evaluating conditional ((step|int == 0 or step|int == 3) and yum_pid_file.stat.exists): 'yum_pid_file' is undefined\n\nThe error appears to be in '/tmp/tripleoxt_690av/overcloud/Controller/update_tasks_step0.yaml': line 18, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n - not (skip_rhel_enforcement | bool)\n- fail: msg=\"ERROR existing yum.pid detected - can't continue! Please ensure there\n ^ here\n"

2020-07-10 01:57:36 | }

2020-07-10 01:57:36 | 2020-07-10 01:57:36.702314 | bc764e10-263d-604f-48bc-000000000036 | TASK | ==> Error during update step 0 which is 'validations' in role 'Controller'

2020-07-10 01:57:36 | 2020-07-10 01:57:36.704007 | bc764e10-263d-604f-48bc-000000000032 | TIMING | Exit if existing yum process | 0:00:31.910 | 0.14s

2020-07-10 01:57:36 | 2020-07-10 01:57:36.763445 | bc764e10-263d-604f-48bc-000000000036 | FATAL | ==> Error during update step 0 which is 'validations' in role 'Controller' | centos-8-rax-ord-0018017813 | error={

2020-07-10 01:57:36 | "changed": false,

2020-07-10 01:57:36 | "msg": "This happened during 'Exit if existing yum process' task in file /tmp/tripleoxt_690av/overcloud/Controller/update_tasks_step0.yaml for host centos-8-rax-ord-0018017813.localdomain.\n"

2020-07-10 01:57:36 | }

2020-07-10 01:57:36 |

Here are the tasks that trigger the problem

        - name: Check for existing yum.pid
          stat: path=/run/yum.pid
          register: yum_pid_file
          when: step|int == 0 or step|int == 3
        - name: Exit if existing yum process
          fail: msg="ERROR existing yum.pid detected - can't continue! Please ensure there is no other package update process for the duration of the minor update worfklow. Exiting."
          when: (step|int == 0 or step|int == 3) and yum_pid_file.stat.exists

So in that case it seems the parser get "exit if existing yum process" in update_tasks_step0.yaml but failed to include "Check for existing yum.pid"

My guess is that the former is in update_tasks_step3.yaml

https://zuul.opendev.org/t/openstack/build/b2b7c2e1f39041beac9afbf6911403fe/log/logs/undercloud/home/zuul/overcloud_update_run_Controller.log#1734-1739

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/740465

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/740465
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=5493304944a443dcab74439c522a18bbf62d1d2a
Submitter: Zuul
Branch: master

commit 5493304944a443dcab74439c522a18bbf62d1d2a
Author: Sofer Athlan-Guyot <email address hidden>
Date: Fri Jul 10 15:35:15 2020 +0200

    Properly collect tasks with or conditional in step file.

    Tasks with or conditional (step == 1 or step == 0) were not showing up
    in the step file.

    To make sure we collect the tasks properly we refactor the filter
    logic:
     1. make all conditional a big string with no spaces and no Boolean
     2. use findall to collect all the conditional step
     3. if step found and it does belong to the current step add it.
     4. if step found and it doesn't belong to the current step remove it.
     5. if step condition not found, let strict decide.
     6. if strict remove it else add it (default behavior)

    We also add a unit test related to that filtering.

    Change-Id: I815443fa7cfd54c9884aff751caf701fd77a8f3a
    Closes-Bug: #1887135

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/742676

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/742676
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=4c4ba312487e0be359838bc42442054c5b280f14
Submitter: Zuul
Branch: stable/ussuri

commit 4c4ba312487e0be359838bc42442054c5b280f14
Author: Sofer Athlan-Guyot <email address hidden>
Date: Fri Jul 10 15:35:15 2020 +0200

    Properly collect tasks with or conditional in step file.

    Tasks with or conditional (step == 1 or step == 0) were not showing up
    in the step file.

    To make sure we collect the tasks properly we refactor the filter
    logic:
     1. make all conditional a big string with no spaces and no Boolean
     2. use findall to collect all the conditional step
     3. if step found and it does belong to the current step add it.
     4. if step found and it doesn't belong to the current step remove it.
     5. if step condition not found, let strict decide.
     6. if strict remove it else add it (default behavior)

    We also add a unit test related to that filtering.

    Backport unit tests were adjusted to accommodate the new interface to
    _write_tasks_per_step[1]

    [1] See I2781cae14663094d531ad70c3d412b3153b46612

    Change-Id: I815443fa7cfd54c9884aff751caf701fd77a8f3a
    Closes-Bug: #1887135
    (cherry picked from commit 5493304944a443dcab74439c522a18bbf62d1d2a)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/744142

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/744142
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=280ed7a59f4a8a6d7395ece2ded6e5959481e25e
Submitter: Zuul
Branch: stable/train

commit 280ed7a59f4a8a6d7395ece2ded6e5959481e25e
Author: Sofer Athlan-Guyot <email address hidden>
Date: Fri Jul 10 15:35:15 2020 +0200

    Properly collect tasks with or conditional in step file.

    Tasks with or conditional (step == 1 or step == 0) were not showing up
    in the step file.

    To make sure we collect the tasks properly we refactor the filter
    logic:
     1. make all conditional a big string with no spaces and no Boolean
     2. use findall to collect all the conditional step
     3. if step found and it does belong to the current step add it.
     4. if step found and it doesn't belong to the current step remove it.
     5. if step condition not found, let strict decide.
     6. if strict remove it else add it (default behavior)

    We also add a unit test related to that filtering.

    Backport unit tests were adjusted to accommodate the new interface to
    _write_tasks_per_step[1]

    [1] See I2781cae14663094d531ad70c3d412b3153b46612

    Change-Id: I815443fa7cfd54c9884aff751caf701fd77a8f3a
    Closes-Bug: #1887135
    (cherry picked from commit 5493304944a443dcab74439c522a18bbf62d1d2a)
    (cherry picked from commit 4c4ba312487e0be359838bc42442054c5b280f14)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 11.5.0 release.