rdo2: BMU job(s) failing on installation of missing package (ceph-ansible)

[Matt Young]

RDO Phase 2 bare metal undercloud jobs are failing on pike, still looking at impact beyond pike.

- https://thirdparty.logs.rdoproject.org/jenkins-oooq-pike-rdo_trunk-bmu-haa16-lab-float_nic_with_vlans-121/console.txt.gz#_2018-03-08_20_41_37_350

---

failed: [undercloud] (item=ceph-ansible) => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_args": {
            "allow_downgrade": false,
            "conf_file": null,
            "disable_gpg_check": false,
            "disablerepo": null,
            "enablerepo": null,
            "exclude": null,
            "install_repoquery": true,
            "installroot": "/",
            "list": null,
            "name": [
                "ceph-ansible"
            ],
            "security": false,
            "skip_broken": false,
            "state": "present",
            "update_cache": false,
            "validate_certs": true
        }
    },
    "item": "ceph-ansible",
    "msg": "No package matching 'ceph-ansible' found available, installed or updated",
    "rc": 126,
    "results": [
        "No package matching 'ceph-ansible' found available, installed or updated"
    ]
}

---

The cleaned up (aka "useful") log of yum install that causes the error:

- https://paste.fedoraproject.org/paste/f2-sU9nrmrgPrdhWghudAw

---

This is blocking promotion of pike rdo2

[Matt Young (halcyondude)]

Installation of packages is happening here:

- https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/baremetal-undercloud/packages/tasks/main.yml

[Raoul Scarazzini (rasca)]

After merging this [1] now, we expect a list of packages declared inside the undercloud_rpm_dependencies variable. By default [2] the contents of the var are correct (just "python-tripleoclient"), but then, in each release file, i.e. [3], we add "ceph-ansible".

Why is this happening?

[1] https://review.openstack.org/#/c/550530/
[2] https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/extras-common/defaults/main.yml#L29
[3] https://github.com/openstack/tripleo-quickstart/blob/master/config/release/trunk/queens.yml#L43

[Matt Young (halcyondude)]

(note: link to yum install is not error, is successful install of tripleoclient)

[Matt Young (halcyondude)]

update: we narrowed down the error to be caused by drift/legacy.

The BMU jobs have been using a different repository definition, which is not adding the ceph jewel repos. Raoul is working on updating these jobs.

[Raoul Scarazzini (rasca)]

We did successful tests over the patches created to use the correct repositories, and the problem is not happening anymore, so I'm closing this bug.