[HA fencing plugin] Parallel deployment could cause not all of the STONITH primitives created

Bug #1411603 reported by Bogdan Dobrelya
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Won't Fix
Medium
Bogdan Dobrelya
6.0.x
Won't Fix
Medium
Unassigned

Bug Description

Fuel deploys non-primary controller nodes in Openstack environment in the parallel.
So, the Fencing plugin is being configured by the puppet in parallel as well.
As a result, some of stonith__* primitives and their location constraints could be absent in the Corosync configuration.
There is a workaround:

Once deployment is done, apply puppet manifest for plugin manually at affected controller nodes consequently:
cd /etc/fuel/plugins/ha_fencing-6.0.0/
puppet apply -vd --modulepath=/etc/fuel/plugins/ha_fencing-6.0.0/puppet/modules/ puppet/manifests/site.pp

(Although, the former apply action could be safely done for *any* controller node, as it would change or break nothing - even if the controller node is not affected by this issue)

Note:
You can list the STONITH primitives and check their health by the commands:
pcs stonith
pcs stonith level verify

And the location constraints could be shown by the commands:
pcs constraint list
pcs constraint ref <stonith_resource_name>

It is expected that every stonith_* primitive should have one "prohibit" and one "allow" location shown by the ref command.

Changed in fuel:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Bogdan Dobrelya (bogdando)
milestone: none → 6.0.1
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-plugins (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/147816

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-plugins (master)

Reviewed: https://review.openstack.org/147816
Committed: https://git.openstack.org/cgit/stackforge/fuel-plugins/commit/?id=55d6be53a99c65a91a97c1db96ad1749492841a6
Submitter: Jenkins
Branch: master

commit 55d6be53a99c65a91a97c1db96ad1749492841a6
Author: Bogdan Dobrelya <email address hidden>
Date: Fri Jan 16 12:31:23 2015 +0100

    Fixes for HA fencing plugin

    * Document a w/a for a parrallel deploy issue
    * Add known issues section to docs
    * Add missing versioning section to docs
    * Deploy the plugin only at controller nodes

    Related blueprint fencing-in-puppet-manifests
    Related-bug: #1411603

    Change-Id: I26ebb5a9d7dae0723b57d171644b626a7b9e691d
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in fuel:
milestone: 6.0.1 → 6.1
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

The plugin was developed for 6.x releases and was not tested with 6.1, so I guess the milestone should be 6.0.1

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

The 6.1 milestone is invalid as the plugin currently supports only 6.0

Changed in fuel:
status: Triaged → Won't Fix
status: Won't Fix → Invalid
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-plugin-ha-fencing (master)

Fix proposed to branch: master
Review: https://review.openstack.org/241686

Changed in fuel:
status: Invalid → In Progress
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Will be fixed if the plugin will start support the 8.0 version of Fuel

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-plugin-ha-fencing (master)

Reviewed: https://review.openstack.org/241686
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-ha-fencing/commit/?id=7ce61a71bb998e66df7c514dfdae396ec0410a59
Submitter: Jenkins
Branch: master

commit 7ce61a71bb998e66df7c514dfdae396ec0410a59
Author: Bogdan Dobrelya <email address hidden>
Date: Wed Nov 4 17:49:24 2015 +0100

    Make cs_property and cs_fencetopo only for primary

    W/a concurrency issue by configuring corosync
    properties and fence topology only one time,
    for the primary controller

    Closes-bug: #1411603

    Change-Id: Id69d7664e6fee994b1b6b46f4c31903c77960499
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

The fix is only partial

Dmitry Pyzhov (dpyzhov)
no longer affects: fuel/8.0.x
Changed in fuel:
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.