cinder service leftover may break the upgrade of M->N

Bug #1701259 reported by Sofer Athlan-Guyot on 2017-06-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
Unassigned
Newton
Undecided
Unassigned

Bug Description

Hi,

originally seen there https://bugzilla.redhat.com/show_bug.cgi?id=1465776.

One can have leftover service definition in the database which uselessly prevents cinder to restart. The error manifest itself by

"major-upgrade-pacemaker-converge.yaml fails restarting openstack-cinder-scheduler: ServiceTooOld: One of the services is in Liberty version. We do not provide backward compatibility with Liberty now, you need to upgrade"

We should cleanup cinder service to avoid this issue.

Changed in tripleo:
importance: Critical → High

Reviewed: https://review.openstack.org/478922
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=64f789dd85414fff8b48df6335d5cd7e7fb55eb7
Submitter: Jenkins
Branch: stable/newton

commit 64f789dd85414fff8b48df6335d5cd7e7fb55eb7
Author: Sofer Athlan-Guyot <email address hidden>
Date: Thu Jun 29 14:57:08 2017 +0200

    [newton only] Ensure cinder services are cleanup during upgrade.

    It may happen that some leftover services definition stay in the
    database. We don't want to prevent upgrade because of that so we just
    clean them up before going on.

    Cleaning up the database while cinder services are running is not a
    problem as the entry are recreated. But duplicate and leftover /will/
    be erased.

    Closes-Bug: #1701259

    Change-Id: I4ea3138e45aa430a89aad305fbb4b5eeefdfa16e

tags: added: in-stable-newton
Changed in tripleo:
milestone: pike-3 → pike-rc1
Ben Nemec (bnemec) wrote :

This was fixed, but the bot didn't close it because it was only on a stable branch.

Changed in tripleo:
status: Triaged → Fix Released

This issue was fixed in the openstack/tripleo-heat-templates 5.3.1 release.

The fix isn't working properly and the issue remain.

Reviewed: https://review.openstack.org/507188
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=0616f2d0be1d4f7748ce89628ebc6af0c87e2514
Submitter: Jenkins
Branch: stable/newton

commit 0616f2d0be1d4f7748ce89628ebc6af0c87e2514
Author: Sofer Athlan-Guyot <email address hidden>
Date: Mon Sep 25 17:59:32 2017 +0200

    [newton only] Cinder services cleanup should happen when mysql is up

    In a previous patch we attempt to fix a issue with old entries in the
    cinder service database making the upgrade fail[1]. The fix was wrong
    as the cinder-manage command was triggered after the galera shutdown,
    making it useless.

    This patch re-order the pacemaker shutdown process to have a state
    where cinder-volume is stopped and pacemaker is up so that we can run
    cinder-manage service cleanup safely.

    [1] https://review.openstack.org/#/c/478922/

    Closes-Bug: #1701259

    Change-Id: I4ee6799b32820a681ffcbf339effdf86337485f7

This issue was fixed in the openstack/tripleo-heat-templates 5.3.3 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers