[manila] upgrade from baremetal to containers fails

[Tom Barron]

Dustin Schoenbrun 2018-07-23 16:31:04 EDT

Description of problem:
On the upgrade of OSP-12 where the Manila services are un-containerized to OSP-13 where they are supposed to be containerized, the Manila Share service does not become containerized and remains a systemd service. Due to this, the upgrade from OSP-12 to OSP-13 fails.

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-8.0.2-43.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Setup and install OSP-12 with Manila configured as an un-containerized service (for example, with a NetApp backend, with the manila-netapp-config.yaml environment file)
2. Follow the upgrade procedure as outlined in https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/upgrading_red_hat_openstack_platform/ ensuring that the Manila environment file is updated to the containerized version (for example, with a NetApp backend, replace manila-netapp-config.yaml with manila-netapp-config-docker.yaml)
3. Observe that on the controller upgrade step (openstack overcloud upgrade run --roles Controller) that the command fails.
4. Observe that the Manila Share service still shows up in Pacemaker but not with anything indicating containerization.
5. Observe that the Manila Share service is still running as a systemd service on the controller node indicated in the "pcs status" command.

Actual results:
Upgrade fails because the Manila Share service is never containerized.

Output of 'pcs status' command:

[heat-admin@controller-1 ~]$ sudo pcs status
Cluster name: tripleo_cluster
Stack: corosync
Current DC: controller-1 (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
Last updated: Mon Jul 23 18:35:27 2018
Last change: Fri Jul 20 23:02:00 2018 by root via cibadmin on controller-0

12 nodes configured
38 resources configured

Online: [ controller-0 controller-1 controller-2 ]
GuestOnline: [ galera-bundle-0@controller-0 galera-bundle-1@controller-1 galera-bundle-2@controller-2 rabbitmq-bundle-0@controller-0 rabbitmq-bundle-1@controller-1 rabbitmq-bundle-2@controller-2 redis-bundle-0@controller-0 redis-bundle-1@controller-1 redis-bundle-2@controller-2 ]

Full list of resources:

 Docker container set: rabbitmq-bundle [192.168.24.1:8787/rhosp12/openstack-rabbitmq:pcmklatest]
   rabbitmq-bundle-0 (ocf::heartbeat:rabbitmq-cluster): Started controller-0
   rabbitmq-bundle-1 (ocf::heartbeat:rabbitmq-cluster): Started controller-1
   rabbitmq-bundle-2 (ocf::heartbeat:rabbitmq-cluster): Started controller-2
 Docker container set: galera-bundle [192.168.24.1:8787/rhosp12/openstack-mariadb:pcmklatest]
   galera-bundle-0 (ocf::heartbeat:galera): Master controller-0
   galera-bundle-1 (ocf::heartbeat:galera): Master controller-1
   galera-bundle-2 (ocf::heartbeat:galera): Master controller-2
 Docker container set: redis-bundle [192.168.24.1:8787/rhosp12/openstack-redis:pcmklatest]
   redis-bundle-0 (ocf::heartbeat:redis): Master controller-0
   redis-bundle-1 (ocf::heartbeat:redis): Slave controller-1
   redis-bundle-2 (ocf::heartbeat:redis): Slave controller-2
 ip-192.168.24.7 (ocf::heartbeat:IPaddr2): Started controller-0
 ip-10.0.0.105 (ocf::heartbeat:IPaddr2): Started controller-1
 ip-172.17.1.19 (ocf::heartbeat:IPaddr2): Started controller-2
 ip-172.17.1.13 (ocf::heartbeat:IPaddr2): Started controller-0
 ip-172.17.3.10 (ocf::heartbeat:IPaddr2): Started controller-1
 ip-172.17.4.10 (ocf::heartbeat:IPaddr2): Started controller-2
 Docker container set: haproxy-bundle [192.168.24.1:8787/rhosp12/openstack-haproxy:pcmklatest]
   haproxy-bundle-docker-0 (ocf::heartbeat:docker): Started controller-0
   haproxy-bundle-docker-1 (ocf::heartbeat:docker): Started controller-1
   haproxy-bundle-docker-2 (ocf::heartbeat:docker): Started controller-2
 openstack-manila-share (systemd:openstack-manila-share): Started controller-0
 Docker container: openstack-cinder-volume [registry.access.redhat.com/rhosp13/openstack-cinder-volume:pcmklatest]
   openstack-cinder-volume-docker-0 (ocf::heartbeat:docker): Started controller-1

Expected results:
Upgrade should be successful and all Manila services shall be containerized.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/586623

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/586623
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=0811ccbc543a8cac991b10d8cea5498b9616e40f
Submitter: Zuul
Branch: master

commit 0811ccbc543a8cac991b10d8cea5498b9616e40f
Author: Tom Barron <email address hidden>
Date: Fri Jul 27 11:51:47 2018 -0400

    Fix manila update, upgrade, and FFU tasks

    Manila has a similar service architecture to Cinder and the
    latter has received several fixes for update, upgrade, and FFU
    tasks that are missing in Manila.

    Align these tasks in the Manila services with Cinder versions
    of the same.

    Closes-Bug: #1784032

    Change-Id: Ifae95bae859a301e6bcf67910cd9140f51584bb3

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/587982

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/587982
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=2959f0d79382831b9e631eaf29e0b1422cdbbce3
Submitter: Zuul
Branch: stable/queens

commit 2959f0d79382831b9e631eaf29e0b1422cdbbce3
Author: Tom Barron <email address hidden>
Date: Fri Jul 27 11:51:47 2018 -0400

    Fix manila update, upgrade, and FFU tasks

    Manila has a similar service architecture to Cinder and the
    latter has received several fixes for update, upgrade, and FFU
    tasks that are missing in Manila.

    Align these tasks in the Manila services with Cinder versions
    of the same.

    Note: I resolved conflicts in
    docker/services/pacemaker/manila-share.yaml
    due to the big refactor patch [1] in master that was not [2]
    backported to stable/queens. We need this backport nonetheless since
    upgrades and FFU do not work for manila on stable/queens.

    Also, this backport also incorporates a fix to a typo in master [3]
    discovered after the original for this cherry-pick had merged there.

    [1] https://review.openstack.org/#/c/562294
    [2] https://review.openstack.org/#/c/576520
    [3] https://review.openstack.org/#/c/588407/

    Closes-Bug: #1784032

    Change-Id: Ifae95bae859a301e6bcf67910cd9140f51584bb3
    (cherry picked from commit 0811ccbc543a8cac991b10d8cea5498b9616e40f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.6 release.