Upgrade fails to bootstrap grafana when monasca enabled

[Scott Shambarger]

Upgrading from Victoria to Wallaby with Monasca enabled fails waiting for Grafana to start.

Grafana defaults to enabled if Monasca is enabled in Wallaby (was default disabled in Victoria), so the database user and tables don't necessarily exist prior to the upgrade.

Running `kolla-ansible upgrade` then attempts to start the Grafana container, but fails to include the bootstrap task (which creates the db) so the container is unable to start with a database connection error, from the grafana container logs:

  service init failed: failed to check table existence: Error 1045: Access denied for user 'grafana'@'xxx' (using password: YES)

I'm guessing normally bootstrap isn't required on upgrade, but since the enabled state for Grafana is changed during upgrade, it is in this case.

I added `import_tasks: bootstrap.yml` to upgrade.yml, and the upgrade completed without error.

[Mark Goddard (mgoddard)]

Good catch Scott.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/kolla-ansible/+/795072

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kolla-ansible/+/795072
Committed: https://opendev.org/openstack/kolla-ansible/commit/12dc474830b3a6b1e8687829cca3fd3963c67b41
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 12dc474830b3a6b1e8687829cca3fd3963c67b41
Author: Mark Goddard <email address hidden>
Date: Mon Jun 7 10:34:36 2021 +0100

    grafana: add bootstrap during upgrade

    Upgrading from Victoria to Wallaby with Monasca enabled fails waiting
    for Grafana to start.

    Grafana defaults to enabled if Monasca is enabled in Wallaby (was
    default disabled in Victoria), so the database user and tables don't
    necessarily exist prior to the upgrade.

    Running `kolla-ansible upgrade` then attempts to start the Grafana
    container, but fails to include the bootstrap task (which creates the
    db) so the container is unable to start with a database connection
    error, from the grafana container logs:

      service init failed: failed to check table existence: Error 1045:
      Access denied for user 'grafana'@'xxx' (using password: YES)

    This change performs a bootstrap of Grafana during upgrade. It is
    required only for the Wallaby release where the default change is
    introduced.

    Closes-Bug: #1931037
    Co-Authored-By: Scott Shambarger

    Change-Id: Ib32b78cfd6f3fe535d30d8aa48f3187b9aec746e

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla-ansible 12.0.0.0rc2 release candidate.