Comment 5 for bug 1867358

Revision history for this message
Ian Marsh (drulgaard) wrote :

(Before I start... confession ... I currently know very little about pacemaker or the inner workings of charms... so apologies if I'm barking up the wrong tree)

It looks like it fails to set up the resources (e.g. the vip):
ubuntu@juju-a55f84-2-lxd-8:~$ sudo crm config show
node 1000: juju-a55f84-2-lxd-8
node 1001: juju-a55f84-0-lxd-8
node 1002: juju-a55f84-1-lxd-8
property cib-bootstrap-options: \
        have-watchdog=false \
        dc-version=1.1.18-2b07d5c5a9 \
        cluster-infrastructure=corosync \
        cluster-name=debian \
        no-quorum-policy=stop \
        cluster-recheck-interval=60 \
        stonith-enabled=false
rsc_defaults rsc-options: \
        resource-stickiness=100 \
        failure-timeout=0

Relevant bit of /var/log/juju/unit-manila-ha-0.log (manila-ha/0 is the leader):
2020-03-26 15:47:43 INFO juju-log hanode:108: Setting cluster symmetry
2020-03-26 15:47:43 WARNING juju-log hanode:108: Inconsistent or absent enable-resources setting []
2020-03-26 15:47:43 WARNING juju-log hanode:108: Unable to calculated desired symmetric-cluster setting
2020-03-26 15:47:43 DEBUG juju-log hanode:108: Deleting Resources
2020-03-26 15:47:43 DEBUG juju-log hanode:108: Configuring Resources: {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Groups: {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Master/Slave (ms): {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Orders: {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Clones: {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Colocations: {}
2020-03-26 15:47:44 DEBUG juju-log hanode:108: Configuring Locations: {}
2020-03-26 15:47:44 INFO juju-log hanode:108: Configuring any remote nodes
2020-03-26 15:47:45 DEBUG juju-log hanode:108: Checking for pacemaker-remote nodes
2020-03-26 15:47:45 WARNING juju-log hanode:108: Inconsistent or absent enable-resources setting []
2020-03-26 15:47:45 WARNING juju-log hanode:108: Unable to calculate whether resources should run on remotes
2020-03-26 15:47:56 DEBUG juju-log hanode:108: Pacemaker is ready
2020-03-26 15:49:34 DEBUG juju-log Pacemaker is ready

Following the code...
unit-manila-ha-0/charm/hooks/hooks.py:ha_relation_changed()
unit-manila-ha-0/charm/hooks/utils.py:set_cluster_symmetry()
unit-manila-ha-0/charm/hooks/utils.py:need_resources_on_remotes()
... it seems like this:
    for relid in relation_ids('pacemaker-remote'):
        for unit in related_units(relid):
            data = parse_data(relid, unit, 'enable-resources')
            # parse_data returns {} if key is absent.
            if type(data) is bool:
                responses.append(data)
... doesn't result in a clean response.

I'll keep digging but in the meantime I'm hoping this triggers an "ah ha" moment in someone more knowledgeable.