Prometheus task "Copying over prometheus config file" has been broken since 10.0.0.0rc1

[Justinas Balciunas]

Found in: kolla-ansible Ussuri 10.0.0.0rc2.dev15.

Problem: when Prometheus is enabled, a prometheus.yml file needs to be generated from the inventory and host facts for prometheus-server. No overrides are being used for the Prometheus role.

TASK [prometheus : Copying over prometheus config file] runs successfully, however, the contents of the produced prometheus.yml file located on the monitoring nodes are: {}

E.g.
root@z141-ost-ctrl1:/etc/kolla/prometheus-server# docker exec -it prometheus_server /bin/bash
(prometheus-server)[prometheus@z141-ost-ctrl1 /]$ cd /etc/prometheus/
(prometheus-server)[prometheus@z141-ost-ctrl1 prometheus]$ ls
prometheus.yml
(prometheus-server)[prometheus@z141-ost-ctrl1 prometheus]$ cat prometheus.yml
{}

As if there was no attempt to render the jinja template at roles/prometheus/templates/prometheus.yml.j2 at all.

After further investigation the reason was found to be the commit 956a29f (https://github.com/openstack/kolla-ansible/commit/956a29f83a6936a1445c7b5212e2b9e9f84a8bc5).

Reverting the previous version of the task "Copying over prometheus config file" (as in stable/train, for instance) produces the output properly and correctly.

[Radosław Piliszek (yoctozepto)]

Indeed it is broken. I've also found other issues with the current approach. I will propose a proper patch soon. (backport to Ussuri)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/735529

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/735529
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=d86f8ab9fe4b68a4b6407740aab064f13cd9ee4c
Submitter: Zuul
Branch: master

commit d86f8ab9fe4b68a4b6407740aab064f13cd9ee4c
Author: Radosław Piliszek <email address hidden>
Date: Mon Jun 15 10:41:57 2020 +0200

    Fix issues with Prometheus config generation

    related to newly introduced merge mechanism.
    1) Per-host overrides cannot be run_once.
    2) Since merge_yaml is silent about missing files, it ignored
       the fact that no proper file was given due to wrong variable
       being referenced (see the closed bug).

    Change-Id: I6db4af4c6e3364838bdae510f300038b0c1560b0
    Closes-Bug: #1882460

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/736300

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/736300
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=6f1bd3e35b46b8c4feb179e697348a5a0efd1549
Submitter: Zuul
Branch: stable/ussuri

commit 6f1bd3e35b46b8c4feb179e697348a5a0efd1549
Author: Radosław Piliszek <email address hidden>
Date: Mon Jun 15 10:41:57 2020 +0200

    Fix issues with Prometheus config generation

    related to newly introduced merge mechanism.
    1) Per-host overrides cannot be run_once.
    2) Since merge_yaml is silent about missing files, it ignored
       the fact that no proper file was given due to wrong variable
       being referenced (see the closed bug).

    Change-Id: I6db4af4c6e3364838bdae510f300038b0c1560b0
    Closes-Bug: #1882460
    (cherry picked from commit d86f8ab9fe4b68a4b6407740aab064f13cd9ee4c)