Queens: minor update fails if docker_config_scripts.yaml doesn't exist

[Emilien Macchi]

Operator failing to run "openstack overcloud update converge" command to finalize minor upgrade procedure. We can see the following errors for failed resources:

  deploy_stdout: |

    PLAY [localhost] ***************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [localhost]

    TASK [Create and ensure setype for /var/log/containers directory] **************
    ok: [localhost]

    TASK [Create /var/lib/tripleo-config directory] ********************************
    changed: [localhost]

    TASK [Check if puppet step_config.pp manifest exists] **************************
    ok: [localhost -> localhost]

    TASK [Set fact when file existed] **********************************************
    skipping: [localhost]

    TASK [Write the puppet step_config manifest] ***********************************
    ok: [localhost]

    TASK [Create /var/lib/docker-puppet] *******************************************
    changed: [localhost]

    TASK [Check if docker-puppet puppet_config.yaml configuration file exists] *****
    ok: [localhost -> localhost]

    TASK [Set fact when file existed] **********************************************
    skipping: [localhost]

    TASK [Write docker-puppet.json file] *******************************************
    ok: [localhost]

    TASK [Create /var/lib/docker-config-scripts] ***********************************
    ok: [localhost]

    TASK [Clean old /var/lib/docker-container-startup-configs.json file] ***********
    ok: [localhost]

    TASK [Check if docker_config_scripts.yaml file exists] *************************
    ok: [localhost -> localhost]

    TASK [Set fact when file existed] **********************************************
    skipping: [localhost]

    TASK [Write docker config scripts] *********************************************
    fatal: [localhost]: FAILED! => {"failed": true, "msg": "Unexpected failure in finding the lookup named '{{ role_data_docker_config_scripts | dictsort }}' in the available lookup plugins"}
     to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/3e395a00-6336-48ba-a4df-664a15b9150d_playbook.retry

    PLAY RECAP *********************************************************************
    localhost : ok=11 changed=2 unreachable=0 failed=1

It looks related to this backport:
https://github.com/openstack/tripleo-heat-templates/commit/ace57871fa6de7d482baf1c41f278459556da108

It's now using dictsort filter which requires a valid dictionary as input. The problem is that role_data_docker_config_scripts is empty if stat_docker_config_scripts doesn't exist. It didn't fail before because we weren't using that filter.

It was fixed after Queens with https://review.opendev.org/#/c/561011/ with:

  role_data_container_config_scripts: "{{ lookup('file', tripleo_role_name + '/container_config_scripts.yaml', errors='ignore')| default({}, True) | from_yaml }}"

Which provided a default (empty hash).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/662756

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/662756
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=073851d32873e8ba54a58594846c6e65d0e9a3ff
Submitter: Zuul
Branch: stable/queens

commit 073851d32873e8ba54a58594846c6e65d0e9a3ff
Author: Emilien Macchi <email address hidden>
Date: Mon Jun 3 09:31:27 2019 -0400

    [queens-only] Write docker config scripts only if config exists

    Do not try to write docker config scripts if the
    docker_config_scripts.yaml doesn't exist.

    I832a2ec34f4ed4a87e30d0c88f4c60bcf2f4c151 was reverted so there is no
    default hash provided to the loop introduced in
    I1fc7431dfc662212b6ca64f4f738760f25b0c30b so the simplest patch
    (one-liner) is to run the task only when the file exist (like the
    previous fact which sets the var used).

    Another way to fix it would have been to revert the revert but it has
    proven to break some CI jobs, so this is the simplest way to fix it now.

    Change-Id: I67da243a7543dcea7af16e964810c4c32e8ecbe1
    Closes-Bug: #1831455

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.4.0 release.