Cannot update pcmkr resource: res_ceph-radosgw_public_hostname

[Jason Hobbs]

This is on a dns HA run of foundation cloud.

hacluster-radosgw/0 had a failed hanode-relation-changed hook with this error:

http://paste.ubuntu.com/p/M46XvpbNrf/

This is with juju 2.4.3

bundles and overlays:
http://paste.ubuntu.com/p/QX44nbqGHM/

[James Page (james-page)]

I suspect this has something todo with:

commit 02d83b2e4ead28c0f144f2e7495e0352843bf39e
Author: Felipe Reyes <email address hidden>
Date: Wed Jun 6 19:38:01 2018 -0400

    Support update parameters of a resource

    This patch implements support to update parameters of an already
    existing resource using "crm configure load update FILE"

    Change-Id: I22730091d674145db4a1187b0904d9f88d9d8c6d
    Partial-Bug: #1753432

which was introduced since the last release.

[James Page (james-page)]

Aug 30 03:29:59 juju-a8e0e0-2-lxd-1 cib[73570]: error: No modify match for /cib/configuration/resources/primitive[@id='res_ceph-radosgw_public_hostname']/instance_attributes[@id='res_ceph-radosgw_public_hostname-instance_attributes']/nvpair[@id='res_ceph-radosgw_public_hostname-instance_attributes-ip_address'] in 0x55d722588f50
Aug 30 03:29:59 juju-a8e0e0-2-lxd-1 cib[73570]: warning: Completed cib_apply_diff operation for section 'all': Application of an update diff failed (rc=-206, origin=juju-a8e0e0-2-lxd-1/cibadmin/2, version=0.15.0)
Aug 30 03:29:59 juju-a8e0e0-2-lxd-1 crmd[73575]: notice: Operation res_ceph-radosgw_public_hostname_stop_0: ok (node=juju-a8e0e0-2-lxd-1, call=21, rc=0, cib-update=17, confirmed=true)

[Felipe Reyes (freyes)]

Hi Jason,

Could you provide us the output of "crm configure show"?

Thanks,

[Jason Hobbs (jason-hobbs)]

No - probably not. This is an automated CI setup and this happens
sporadically. I would suggest adding something to the charm to collect that
information.

On Mon, Sep 3, 2018 at 10:40 AM Felipe Reyes <email address hidden>
wrote:

> Hi Jason,
>
> Could you provide us the output of "crm configure show"?
>
> Thanks,
>
> ** Changed in: charm-hacluster
> Assignee: (unassigned) => Felipe Reyes (freyes)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1789915
>
> Title:
> Cannot update pcmkr resource: res_ceph-radosgw_public_hostname
>
> Status in OpenStack hacluster charm:
> Confirmed
>
> Bug description:
> This is on a dns HA run of foundation cloud.
>
> hacluster-radosgw/0 had a failed hanode-relation-changed hook with
> this error:
>
> http://paste.ubuntu.com/p/M46XvpbNrf/
>
> This is with juju 2.4.3
>
> bundles and overlays:
> http://paste.ubuntu.com/p/QX44nbqGHM/
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-hacluster/+bug/1789915/+subscriptions
>

[Felipe Reyes (freyes)]

the function crm_opt_exists() took a naive approach, just checking if the resource appears in the outpug "crm configure show" is too naive. When dns-ha is configured a group of resources is created, so making the string appear twice, one in the definition of the resource itself and another time for the group definition https://github.com/openstack/charm-ceph-radosgw/blob/master/hooks/charmhelpers/contrib/openstack/ha/utils.py#L201

crm_opt_exists() needs to be rewritten to use a different approach. This function should use "crm configure show xml" and parse the xml.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/599471

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/599471
Committed: https://git.openstack.org/cgit/openstack/charm-hacluster/commit/?id=1fdeb0b3e0deac975d944db531477a564debb287
Submitter: Zuul
Branch: master

commit 1fdeb0b3e0deac975d944db531477a564debb287
Author: Felipe Reyes <email address hidden>
Date: Mon Sep 3 22:23:49 2018 -0300

    Use crm configure show xml to find defined resources

    Using the xml output provided by "crm configure" and parsing it to look
    for nodes that match the xpath ".//*[@id='$NAME']". The test case added
    uses the xml generated when ceph-radosgw has dns-ha enabled which
    creates a groups of hostnames that cross references resources making the
    previous approach give false positives.

    Change-Id: If1c3584c889e7e101f15ed5ba6de89c687667754
    Closes-Bug: 1789915