VIPs not monitored

Bug #1801741 reported by Andrea Ieri
26
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Keystone Charm
Fix Released
High
Liam Young

Bug Description

As of hacluster:49, virtual IPs do not get a monitor action and are thus not monitored.
A sample VIP definition currently looks as follows:

<primitive id="res_ks_eth0_vip" class="ocf" provider="heartbeat" type="IPaddr2">
  <instance_attributes id="res_ks_eth0_vip-instance_attributes">
     <nvpair name="ip" value="10.10.10.50" id="res_ks_eth0_vip-instance_attributes-ip"/>
     <nvpair name="cidr_netmask" value="255.255.254.0" id="res_ks_eth0_vip-instance_attributes-cidr_netmask"/>
     <nvpair name="nic" value="eth0" id="res_ks_eth0_vip-instance_attributes-nic"/>
  </instance_attributes>
</primitive>

Since no monitor action is defined, pacemaker cannot notice if the VIP has gone missing.
Additionally, the VIP is not even reinstated after the recheck interval (currently 15 minutes), possibly because only resources with monitor actions are actually probed.
A manual resource reprobe reinstates the virtual IP.

How to reproduce:
Manually remove a virtual IP (ip a d / ifdown / etc)

Expected behavior:
Pacemaker quickly recovers and reinstates the virtual IP if possible (either in place, or on a cluster neighbor)

Current behavior:
No action

Revision history for this message
Liam Young (gnuoy) wrote :

This is a nasty bug. As Andrea says the losing the vip does not trigger a failover.

Changed in charm-hacluster:
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Liam Young (gnuoy) wrote :

Looks like this is actually a bug in the principle charms as they pass ocf definitions to the hacluster charm down the 'ha' relation.

Revision history for this message
Andrea Ieri (aieri) wrote :

I've filed lp#1805450 for keystone, further bugs could be filed once a solution has been defined.
I suppose this one could then be closed.

Revision history for this message
James Page (james-page) wrote :

linking to bug 1805450

James Page (james-page)
Changed in charm-hacluster:
status: Confirmed → In Progress
assignee: nobody → Liam Young (gnuoy)
affects: charm-hacluster → charm-keystone
Revision history for this message
Liam Young (gnuoy) wrote :

VIP monitoring has now been added to both the hacluster interface https://github.com/openstack/charm-interface-hacluster/commit/ab38cf8b072b45e5fd33c1e88544c9a8ae7a951d (for reactive charms) and charm-helpers https://github.com/juju/charm-helpers/pull/257 (for traditional charms). Reactive charms will need a rebuild and traditional charms will need a charmhelper sync

Revision history for this message
Liam Young (gnuoy) wrote :

atm this will only add vip monitoring to new deployments as the hacluster charm does not update existing hacluster resources when parameters change.

Alvaro Uria (aluria)
tags: added: canonical-bootstack
Revision history for this message
Peter Sabaini (peter-sabaini) wrote :

For clarification, is the plan to fix up resource monitoring on charm upgrade?

[subscribed field-medium]

Revision history for this message
Peter Sabaini (peter-sabaini) wrote :

Also, am I correct in thinking this will affect all charms that make use of the "ha" rel, not only charm-keystone?

Frode Nordahl (fnordahl)
Changed in charm-keystone:
status: In Progress → Fix Committed
James Page (james-page)
Changed in charm-keystone:
milestone: none → 19.04
David Ames (thedac)
Changed in charm-keystone:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.