OCF: Use private attributes to not trigger pacemaker transitions on each update

Bug #1524672 reported by Bogdan Dobrelya
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Medium
Liubov Efremova
Mitaka
Won't Fix
Medium
Unassigned

Bug Description

According to Andrew's comments https://review.openstack.org/#/c/248319 crm_attribute does not support private attributes for corosync nodes. Each update with crm_attribute triggers unnecessary transitions. We should use private attributes for nodes when possible.

That means that "crm_attribute -N <node>" should be replaced to private attributes with "attrd_updater", see examples
https://review.openstack.org/#/c/248319/1/ocf/NovaEvacuate
https://review.openstack.org/#/c/248627/1/ocf/NovaEvacuate

Note, for some cases other nodes should be able to see attributes of each other. For such cases, private attributes should not be used.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

I'm not sure of the impact of this issue. Is it so bad to have extra transitions? Maybe not, but still it'd be better not having them for sure. That is why I put the medium prio

Changed in fuel:
importance: Undecided → Medium
assignee: nobody → Fuel Library Team (fuel-library)
milestone: none → 8.0
tags: added: area-library
description: updated
description: updated
description: updated
summary: - OCF: Use private attributes to not trigger peacemaker transitions on
- each update
+ OCF: Use private attributes to not trigger pacemaker transitions on each
+ update
Changed in fuel:
status: New → Confirmed
tags: added: team-bugfix
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
milestone: 8.0 → 9.0
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Liubov Efremova (lefremova)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/281202

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Liubov Efremova (lefremova) wrote :

I marked the previous fix as 'Partial-Bug: #1524672', but after further investigations I suppose it closes the current issue.

Node attributes are also created in 'fuel-library/files/fuel-ha-utils/ocf/mysql-wss' but we shouldn't change them to private:
        -'galera_master_timeout' is a cluster option
        -'gtid' should be public as we are getting its value in 'get_possible_masters()' method for all online nodes in cluster.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/281202
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=9a3aa4b709138e25d1f0233c1d5ee22644924b21
Submitter: Jenkins
Branch: master

commit 9a3aa4b709138e25d1f0233c1d5ee22644924b21
Author: Liubov Efremova <email address hidden>
Date: Wed Feb 17 13:38:02 2016 +0300

    Private attributes usage in rabbitmq script

    There are three types of rabbitmq attributes for pacemaker nodes:
     -'rabbit-master'
     -'rabbit-start-time'
     - timeouts:
      -'rabbit_list_channels_timeouts'
      -'rabbit_get_alarms_timeouts'
      -'rabbit_list_queues_timeouts'

    Attributes with names 'rabbit-master' and 'rabbit-start-time' should be
    public because we monitor this attributes in cycle for all nodes in our
    script.

    All timeouts attributes were changed to private to avoid unnecessary
    transitions.

    Also, --lifetime and --node options were removed for attrd_updater as
    'lifetime' for this command is always 'reboot' and 'node' default value
    is local one.

    Closes-bug: #1524672
    Change-Id: Ie45ae3a82b8daa35dbdd977dc894877160af457b

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Changed in fuel:
status: Fix Committed → Confirmed
Revision history for this message
Liubov Efremova (lefremova) wrote :

The current version of Pacemaker is 1.1.12. The command 'attrd_updater' have '-p' option only since 1.1.13. It's necessary to update the version of Pacemaker at first and then apply the previous commit again.

Revision history for this message
Bug Checker Bot (bug-checker) wrote : Autochecker

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

version

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

tags: added: need-info
tags: added: keep-in-9.0
Changed in fuel:
assignee: Liubov Efremova (lefremova) → MOS Linux (mos-linux)
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

MOS Linux team, please investigate the effort required to raise the version of pacemaker to add this capability. It will reduce the amount of transitions in Corosync.

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/trusty/pacemaker (master)

Fix proposed to branch: master
Change author: Ivan Suzdal <email address hidden>
Review: https://review.fuel-infra.org/19059

Changed in fuel:
status: Confirmed → In Progress
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: MOS Linux (mos-linux) → Ivan Suzdal (isuzdal)
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to packages/trusty/pacemaker (master)

Reviewed: https://review.fuel-infra.org/19059
Submitter: Pkgs Jenkins <email address hidden>
Branch: master

Commit: 41b68cf5f13e511373eb1a102345b0119fc198ae
Author: Ivan Suzdal <email address hidden>
Date: Fri Apr 1 14:29:37 2016

Bump pacemaker version to 1.1.14

Backport from xenial

Change-Id: Ia260241505d2cb32f64f293e7072e6931e78a523
Closes-Bug: #1524672

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to packages/trusty/pacemaker (9.0)

Fix proposed to branch: 9.0
Change author: Ivan Suzdal <email address hidden>
Review: https://review.fuel-infra.org/19178

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to packages/trusty/pacemaker (9.0)

Reviewed: https://review.fuel-infra.org/19178
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: a4705a31308855d23b62d33dad21ff28b8c34adf
Author: Ivan Suzdal <email address hidden>
Date: Tue Apr 5 10:28:55 2016

Bump pacemaker version to 1.1.14

Backport from xenial

Change-Id: Ia260241505d2cb32f64f293e7072e6931e78a523
Closes-Bug: #1524672
(cherry picked from commit 41b68cf5f13e511373eb1a102345b0119fc198ae)

Revision history for this message
Alexey Lebedeff (alebedev-a) wrote :

'crmsh' package is broken after this upgrade - pacemaker 1.1.14 compatibilty was added only in crmsh 2.1.5

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to packages/trusty/crmsh (master)

Reviewed: https://review.fuel-infra.org/19739
Submitter: Pkgs Jenkins <email address hidden>
Branch: master

Commit: 6e68a7c7bdf0f52cfef29e79108d06b9f037e7e3
Author: Ivan Suzdal <email address hidden>
Date: Mon Apr 18 13:55:40 2016

Bump crmsh package to 2.1.5

Rebuilt from upstream tag 2.1.5

Related-Bug: #1524672
Related-Bug: #1567864

Change-Id: Iabe3e5479fa9196bc10346ed762899d8ca82d8f0

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to packages/trusty/crmsh (9.0)

Related fix proposed to branch: 9.0
Change author: Ivan Suzdal <email address hidden>
Review: https://review.fuel-infra.org/19783

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to packages/trusty/crmsh (9.0)

Reviewed: https://review.fuel-infra.org/19783
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: 7b51b3cba8d3f204c412242b09d1f20bd9998721
Author: Ivan Suzdal <email address hidden>
Date: Mon Apr 18 18:14:57 2016

Bump crmsh package to 2.1.5

Rebuilt from upstream tag 2.1.5

Related-Bug: #1524672
Closes-Bug: #1567864

Change-Id: Iabe3e5479fa9196bc10346ed762899d8ca82d8f0
(cherry picked from commit 6e68a7c7bdf0f52cfef29e79108d06b9f037e7e3)

Revision history for this message
Alexander Gubanov (ogubanov) wrote :

Verified on MOS 9.0 (build 232):
root@node-1:~# dpkg -l | grep crmsh
ii crmsh 2.1.5-1~u14.04+mos1 amd64 CRM shell for the pacemaker cluster manager

Moved to "Fix Release"

Changed in fuel:
status: Fix Committed → Fix Released
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Note, updated Pacemaker only enables the real fix to be applied, which is in progress (https://review.openstack.org/#/c/314633/), reopenting

Changed in fuel:
status: Fix Released → Won't Fix
no longer affects: fuel/newton
Changed in fuel:
milestone: 9.0 → 10.0
assignee: Ivan Suzdal (isuzdal) → Liubov Efremova (lefremova)
status: Won't Fix → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/314633
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=f7f0f17ffe6d65b7919f982c5a2bfb8f3746e402
Submitter: Jenkins
Branch: master

commit f7f0f17ffe6d65b7919f982c5a2bfb8f3746e402
Author: Liubov Efremova <email address hidden>
Date: Tue May 10 15:27:53 2016 +0000

    Private attributes usage in rabbitmq script

    There are three types of rabbitmq attributes for pacemaker nodes:
     -'rabbit-master'
     -'rabbit-start-time'
     - timeouts:
      -'rabbit_list_channels_timeouts'
      -'rabbit_get_alarms_timeouts'
      -'rabbit_list_queues_timeouts'

    Attributes with names 'rabbit-master' and 'rabbit-start-time' should be
    public because we monitor this attributes in cycle for all nodes in our
    script.

    All timeouts attributes were changed to private to avoid unnecessary
    transitions.

    Also, --lifetime and --node options were removed for attrd_updater as
    'lifetime' for this command is always 'reboot' and 'node' default value
    is local one.

    This reverts commit b2b191d2e28b96c9f9a6ea440a383cf4f691d8ad.
    (As the pacemaker version was updated).

    Closes-bug: #1524672

    Change-Id: I6f0d4a99641b847321754d75605a78fbbc96ddad

Changed in fuel:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.