Backup fails because of sysinv-utils

[Mihnea Saracin]

Brief Description
-----------------
I think this issue was introduced by https://opendev.org/starlingx/config/commit/65dfc46b9b0f08e9515970268c50fa5027d4c442.

On my system configuration there was no proxy present and the sysinv-utils script fails to treat this case. Moreover, the script returns 0 when it shouldn't and the playbook fails to catch the error.

2019-11-22 16:17:54,405 p=1187653 u=sysadmin | TASK [backup/backup-system : Create the override backup file] ***********************************************************************************************
2019-11-22 16:17:54,405 p=1187653 u=sysadmin | task path: /usr/share/ansible/stx-ansible/playbooks/roles/backup/backup-system/tasks/main.yml:228
2019-11-22 16:17:54,496 p=1187653 u=sysadmin | Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
2019-11-22 16:18:00,053 p=1187653 u=sysadmin | changed: [localhost] => {
    "changed": true,
    "cmd": [
        "/usr/bin/sysinv-utils",
        "create-host-overrides",
        "/opt/backups/ansible.LIi4FG/override/localhost_override_backup.yml"
    ],
    "delta": "0:00:05.286237",
    "end": "2019-11-22 16:18:00.005991",
    "failed_when_result": false,
    "invocation": {
        "module_args": {
            "_raw_params": "/usr/bin/sysinv-utils create-host-overrides /opt/backups/ansible.LIi4FG/override/localhost_override_backup.yml",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "start": "2019-11-22 16:17:54.719754",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "sysinv 2019-11-22 16:17:59.775 1202266 ERROR sysinv.cmd.utils [-] Resource could not be found.: NotFound: Resource could not be found.\u001b[00m",
    "stdout_lines": [
        "sysinv 2019-11-22 16:17:59.775 1202266 ERROR sysinv.cmd.utils [-] Resource could not be found.: NotFound: Resource could not be found.\u001b[00m"
    ]
}

Because the sysinv-utils script does not create the localhost_override_backup.yml file, the playbook generates an error later when a task tries to write to the yml file.

2019-11-22 16:18:35,679 p=1187653 u=sysadmin | TASK [backup/backup-system : Append registries configuration] ***********************************************************************************************
2019-11-22 16:18:35,679 p=1187653 u=sysadmin | task path: /usr/share/ansible/stx-ansible/playbooks/roles/backup/backup-system/tasks/main.yml:240
2019-11-22 16:18:36,157 p=1187653 u=sysadmin | Using module file /usr/lib/python2.7/site-packages/ansible/modules/files/blockinfile.py
2019-11-22 16:18:36,495 p=1187653 u=sysadmin | fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": false,
            "block": "docker_registries:\n docker.elastic.co: {}\n docker.io: {}\n gcr.io: {}\n k8s.gcr.io: {}\n quay.io: {}\n",
            "content": null,
            "create": false,
            "delimiter": null,
            "directory_mode": null,
            "follow": false,
            "force": null,
            "group": null,
            "insertafter": null,
            "insertbefore": null,
            "marker": "",
            "marker_begin": "BEGIN",
            "marker_end": "END",
            "mode": null,
            "owner": null,
            "path": "/opt/backups/ansible.LIi4FG/override/localhost_override_backup.yml",
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": null,
            "state": "present",
            "unsafe_writes": null,
            "validate": null
        }
    },
    "msg": "Path /opt/backups/ansible.LIi4FG/override/localhost_override_backup.yml does not exist !",
    "rc": 257
}

Severity
--------
Critical

Steps to Reproduce
------------------
Run the backup playbook:
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -e "ansible_become_pass=Li69nux* admin_password=Li69nux*"

Expected Behavior
------------------
The playbook should run succesfully

Actual Behavior
----------------
The playbook run failed

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
One node system

Branch/Pull Time/Commit
-----------------------
SW_VERSION="19.09"
BUILD_TARGET="Unknown"
BUILD_TYPE="Informal"
BUILD_ID="n/a"

JOB="n/a"
BUILD_BY="jenkins"
BUILD_NUMBER="n/a"
BUILD_HOST="358e067817c0"
BUILD_DATE="2019-11-22 08:44:28 +0000"

BUILD_DIR="/"
WRS_SRC_DIR="/localdisk/designer/jenkins/test/cgcs-root"
WRS_GIT_BRANCH="HEAD"
CGCS_SRC_DIR="/localdisk/designer/jenkins/test/cgcs-root/stx"
CGCS_GIT_BRANCH="HEAD"

[Kristine Bujold (kbujold)]

This has been fixed with https://review.opendev.org/#/c/695718

[Ghada Khalil (gkhalil)]

This initially missed screening. Given the severity, it's stx.3.0 gating.
I verified that the above fix was sourced in master before the r/stx.3.0 branch creation, so the content should already be in the 3.0 builds

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mihnea Saracin (<email address hidden>) on branch: master
Review: https://review.opendev.org/695891