Cinder backends cause cinder-volume to deploy as both service and container

[Alan Bishop]

Heat templates in puppet/services for several Cinder backends contain a step_config that causes the cinder-volume service to deploy on the baremetal host when the service is supposed to only run in a container. The backend templates should only be generating config_settings (basically, hieradata for each backend), and the CinderVolume template is responsible for the config steps.

Background: the Cinder backends are modelled as composable services, but really they are just responsible for establishing the backend config settings for the CinderVolume service. The backends are implemented in puppet/services, and there are no (and no need for) equivalent templates in docker/services. To deploy a backend, you specify either the docker (containerized) or puppet (baremetal) version of the CinderVolume service, and the puppet template for the backend.

The problem is the backend puppet templates contain a step_config that causes puppet-tripleo to deploy the service. This is benign when CinderVolume is deployed on baremetal because it simply repeats the step_config that's in puppet/services/cinder-volume.yaml. But when deploying cinder in containers, the duplicate step_config causes the service to also be deployed on the baremetal host.

Here is an example of what happens after deploying cinder in containers with puppet/services/cinder-backend-dellsc.yaml included in the deployment.

First, there are too many cinder-volume processes (there should be two):

[root@overcloud-controller-0 ~]# pgrep -a cinder-volume
90620 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/volume.log
90842 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf --logfile /var/log/cinder/volume.log
98518 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf
100194 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf

Two are running in the cinder_volume container:

[root@overcloud-controller-0 ~]# docker top cinder_volume
UID PID PPID C STIME TTY TIME CMD
42407 98518 98491 1 14:57 ? 00:00:15 /usr/bin/p
ython2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf
42407 100194 98518 1 14:58 ? 00:00:16 /usr/bin/p
ython2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf

And two are running on the baremetal host:

[root@overcloud-controller-0 ~]# systemctl status openstack-cinder-volume
● openstack-cinder-volume.service - OpenStack Cinder Volume Server
   Loaded: loaded (/usr/lib/systemd/system/openstack-cinder-volume.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2018-04-30 14:55:49 UTC; 34min ago
 Main PID: 90620 (cinder-volume)
   Memory: 155.3M
   CGroup: /system.slice/openstack-cinder-volume.service
   ├─90620 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf -...
   └─90842 /usr/bin/python2 /usr/bin/cinder-volume --config-file /usr/share/cinder/cinder-dist.conf --config-file /etc/cinder/cinder.conf -...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/565244

[Bogdan Dobrelya (bogdando)]

Note, we prolly need to backport for queens and pike, but not for ocata (containers had yet become a thing in ocata)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/565542

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565244
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=94b5c1155e7b8ccd54f0e17e8fb07f5a7a378aea
Submitter: Zuul
Branch: master

commit 94b5c1155e7b8ccd54f0e17e8fb07f5a7a378aea
Author: Alan Bishop <email address hidden>
Date: Fri Apr 27 14:47:47 2018 -0400

    Remove step_config from CinderVolume backend services

    Cinder backends should not repeat the step_config that the CinderVolume
    service uses to trigger things in puppet-tripleo. This is benign when
    Cinder runs on baremetal, but causes problems when Cinder runs in
    containers. It causes the cinder-volume service to run twice, in both
    the container as well as on the baremetal host.

    Closes-Bug: #1768063
    Change-Id: Ie02553f74563883094a13c67a958b15371e90424

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565542
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=492b66755868ed599672d1bbae2b057f450e7f6f
Submitter: Zuul
Branch: stable/queens

commit 492b66755868ed599672d1bbae2b057f450e7f6f
Author: Alan Bishop <email address hidden>
Date: Fri Apr 27 14:47:47 2018 -0400

    Remove step_config from CinderVolume backend services

    Cinder backends should not repeat the step_config that the CinderVolume
    service uses to trigger things in puppet-tripleo. This is benign when
    Cinder runs on baremetal, but causes problems when Cinder runs in
    containers. It causes the cinder-volume service to run twice, in both
    the container as well as on the baremetal host.

    Closes-Bug: #1768063
    Change-Id: Ie02553f74563883094a13c67a958b15371e90424
    (cherry picked from commit 94b5c1155e7b8ccd54f0e17e8fb07f5a7a378aea)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/569471

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/569471
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=bd2b1ffb7a4573d39dcb364fb900f678bc19cfa4
Submitter: Zuul
Branch: stable/pike

commit bd2b1ffb7a4573d39dcb364fb900f678bc19cfa4
Author: Alan Bishop <email address hidden>
Date: Fri Apr 27 14:47:47 2018 -0400

    Remove step_config from CinderVolume backend services

    Cinder backends should not repeat the step_config that the CinderVolume
    service uses to trigger things in puppet-tripleo. This is benign when
    Cinder runs on baremetal, but causes problems when Cinder runs in
    containers. It causes the cinder-volume service to run twice, in both
    the container as well as on the baremetal host.

    Closes-Bug: #1768063
    Change-Id: Ie02553f74563883094a13c67a958b15371e90424
    (cherry picked from commit 94b5c1155e7b8ccd54f0e17e8fb07f5a7a378aea)
    (cherry picked from commit 492b66755868ed599672d1bbae2b057f450e7f6f)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.0.3 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 7.0.14 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/581763

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/581763
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=dbdde60ebc5e4a78d276bf0a73bd9bca8c2cda74
Submitter: Zuul
Branch: master

commit dbdde60ebc5e4a78d276bf0a73bd9bca8c2cda74
Author: Hamdy Khader <email address hidden>
Date: Wed Jul 11 17:31:14 2018 +0300

    Remove step_config from NVMEoF cinder backend

    Change-Id: I84d6d3eacf5b9458fadbeda88663eb2099f42ff0
    Related-Bug: #1768063