BlockStorage role conflicts with Controller on CinderVolume

Bug #1786412 reported by Michele Baldessari
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Michele Baldessari

Bug Description

When deploying a BlockStorage role with an HA Controller the following error will be observed:
Error: pcs -f /var/lib/pacemaker/cib/puppet-cib-backup20180712-8-1yx6b6z create failed: Error: unable to set attribute cinder-volume-role\",
Error: /Stage[main]/Tripleo::Profile::Pacemaker::Cinder::Volume_bundle/Pacemaker::Property[cinder-volume-role-nodefoo]/Pcmk_property[property-nodefoo-cinder-volume-role]/ensure: change from absent to present failed: pcs -f /var/lib/pacemaker/cib/puppet-cib-backup20180712-8-1yx6b6z create failed: Error: unable to set attribute cinder-volume-role",

This problem exists since pike because:
1)
We have a controller role with CinderVolume which will set the service name to 'cinder_volume', which means the controller nodes will contribute populating the 'cinder_volume_short_node_names' hiera list.

2) Then we have the BlockStorage role which defines the non-ha variant of cinder volume called 'OS::TripleO::Services::BlockStorageCinderVolume' which points to puppet/services/cinder-volume.yaml and, since it also has a service_name 'cinder_volume' all the nodes of BlockStorage role will also populate the 'cinder_volume_short_node_names' hiera key

3) The pacemaker implementation of HA services needs to set a cluster per-node property for each node where cinder-volume is allowed to run and uses the 'cinder_volume_short_node_names' for that. Since BlockStorage is in that list but has no pacemaker service attached to it things break down.

The commit that introduced this is the following one:
    Cinder volume service is not managed by Pacemaker on BlockStorage

    We do not want cinder-volume to be managed by Pacemaker on
    BlockStorage nodes, where Pacemaker is not running at all.

This commit + the fact that since pike we started to set all properties from one node by iterating the cinder_volume_short_node_names hiera key breaks this scenario

Changed in tripleo:
assignee: nobody → Michele Baldessari (michele)
Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
Michele Baldessari (michele) wrote :
Changed in tripleo:
milestone: rocky-rc1 → stein-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (master)

Reviewed: https://review.openstack.org/588148
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=81d4dfa7e0c5d4fcc1cd9fae19b860184c6e351f
Submitter: Zuul
Branch: master

commit 81d4dfa7e0c5d4fcc1cd9fae19b860184c6e351f
Author: Michele Baldessari <email address hidden>
Date: Thu Aug 2 08:36:52 2018 +0200

    Force cinder properties to be set on ly on nodes with pcmk on it

    When using the BlockStorage role there is one cinder volume per node and
    those are not managed by pcmk. So when we force the property for all
    nodes that have cinder_volume we will fail because pcmk is not running
    on those nodes. Let's not set properties for nodes that are not running
    pacemaker.

    While we're at it let's remove *_nodes_count which are not used anyway.

    Closes-Bug: #1786412

    Change-Id: I42e9f3244bad60b5df2dfa940f132f72c606620e

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-tripleo (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/593373

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-tripleo (stable/queens)

Reviewed: https://review.openstack.org/593373
Committed: https://git.openstack.org/cgit/openstack/puppet-tripleo/commit/?id=2fc929de08ff7811c9de0e23c20548d0b17efb65
Submitter: Zuul
Branch: stable/queens

commit 2fc929de08ff7811c9de0e23c20548d0b17efb65
Author: Michele Baldessari <email address hidden>
Date: Thu Aug 2 08:36:52 2018 +0200

    Force cinder properties to be set on ly on nodes with pcmk on it

    When using the BlockStorage role there is one cinder volume per node and
    those are not managed by pcmk. So when we force the property for all
    nodes that have cinder_volume we will fail because pcmk is not running
    on those nodes. Let's not set properties for nodes that are not running
    pacemaker.

    While we're at it let's remove *_nodes_count which are not used anyway.

    Closes-Bug: #1786412

    Change-Id: I42e9f3244bad60b5df2dfa940f132f72c606620e
    (cherry picked from commit 81d4dfa7e0c5d4fcc1cd9fae19b860184c6e351f)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 9.3.0

This issue was fixed in the openstack/puppet-tripleo 9.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/puppet-tripleo 8.3.6

This issue was fixed in the openstack/puppet-tripleo 8.3.6 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.