cinder service leftover may break the upgrade of M->N

[Sofer Athlan-Guyot]

Hi,

originally seen there https://bugzilla.redhat.com/show_bug.cgi?id=1465776.

One can have leftover service definition in the database which uselessly prevents cinder to restart. The error manifest itself by

"major-upgrade-pacemaker-converge.yaml fails restarting openstack-cinder-scheduler: ServiceTooOld: One of the services is in Liberty version. We do not provide backward compatibility with Liberty now, you need to upgrade"

We should cleanup cinder service to avoid this issue.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/478922

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/478922
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=64f789dd85414fff8b48df6335d5cd7e7fb55eb7
Submitter: Jenkins
Branch: stable/newton

commit 64f789dd85414fff8b48df6335d5cd7e7fb55eb7
Author: Sofer Athlan-Guyot <email address hidden>
Date: Thu Jun 29 14:57:08 2017 +0200

    [newton only] Ensure cinder services are cleanup during upgrade.

    It may happen that some leftover services definition stay in the
    database. We don't want to prevent upgrade because of that so we just
    clean them up before going on.

    Cleaning up the database while cinder services are running is not a
    problem as the entry are recreated. But duplicate and leftover /will/
    be erased.

    Closes-Bug: #1701259

    Change-Id: I4ea3138e45aa430a89aad305fbb4b5eeefdfa16e

[Ben Nemec (bnemec)]

This was fixed, but the bot didn't close it because it was only on a stable branch.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.3.1 release.

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

The fix isn't working properly and the issue remain.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/507188
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=0616f2d0be1d4f7748ce89628ebc6af0c87e2514
Submitter: Jenkins
Branch: stable/newton

commit 0616f2d0be1d4f7748ce89628ebc6af0c87e2514
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Sep 25 17:59:32 2017 +0200

    [newton only] Cinder services cleanup should happen when mysql is up

    In a previous patch we attempt to fix a issue with old entries in the
    cinder service database making the upgrade fail[1]. The fix was wrong
    as the cinder-manage command was triggered after the galera shutdown,
    making it useless.

    This patch re-order the pacemaker shutdown process to have a state
    where cinder-volume is stopped and pacemaker is up so that we can run
    cinder-manage service cleanup safely.

    [1] https://review.openstack.org/#/c/478922/

    Closes-Bug: #1701259

    Change-Id: I4ee6799b32820a681ffcbf339effdf86337485f7

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.3.3 release.