Failed to delete old ocf resources by delete_resources option

[Hua Zhang]

Now hacluster charm is using 'crm -w -F resource stop res_ceilometer_agent_central' and 'crm -w -F configure delete res_ceilometer_agent_central' to delete old ocf resource via delete_resources option, but the first step above will fail when ocf file no longer exists (e.g. due to packages no longer being available after upgrade). In addition, the daemon process may still be running after the upgrade in this case.

I found the command 'crm configure delete' can be run to delete old ocf file after running 'crm resource cleanup'.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/384438

[Hua Zhang (zhhuabj)]

This is test result[1] by testing the patch[2] and the patch[3], we can see the expected log and result as below after the ceilometer charm upgrade (ceilometer-agent-central becomes ceilometer-polling).

2016-10-11 03:25:44 DEBUG juju-log ha:43: Cleanuping and deleting resource res_ceilometer_agent_central
2016-10-11 03:25:44 INFO ha-relation-changed Waiting for 1 replies from the CRMd. OK
2016-10-11 03:25:44 INFO ha-relation-changed Cleaning up res_ceilometer_agent_central on juju-zhhuabj-machine-1
2016-10-11 03:25:44 INFO ha-relation-changed Cleaning up res_ceilometer_agent_central on juju-zhhuabj-machine-2
2016-10-11 03:25:44 INFO ha-relation-changed Cleaning up res_ceilometer_agent_central on juju-zhhuabj-machine-3
2016-10-11 03:25:45 DEBUG juju-log ha:43: Configuring Resources: {'res_ceilometer_haproxy': 'lsb:haproxy', 'res_ceilometer_polling': 'ocf:openstack:ceilometer-polling'}

ubuntu@juju-zhhuabj-machine-1:~$ sudo crm status
Last updated: Tue Oct 11 03:30:15 2016
Last change: Tue Oct 11 03:26:17 2016 via crmd on juju-zhhuabj-machine-1
Stack: corosync
Current DC: juju-zhhuabj-machine-2 (1002) - partition with quorum
Version: 1.1.10-42f2063
3 Nodes configured
5 Resources configured
Online: [ juju-zhhuabj-machine-1 juju-zhhuabj-machine-2 juju-zhhuabj-machine-3 ]
 Resource Group: grp_ceilometer_vips
     res_ceilometer_eth0_vip (ocf::heartbeat:IPaddr2): Started juju-zhhuabj-machine-2
 Clone Set: cl_ceilometer_haproxy [res_ceilometer_haproxy]
     Started: [ juju-zhhuabj-machine-1 juju-zhhuabj-machine-2 juju-zhhuabj-machine-3 ]
 res_ceilometer_polling (ocf::openstack:ceilometer-polling): Started juju-zhhuabj-machine-1

[1] http://pastebin.ubuntu.com/23306410/
[2] https://review.openstack.org/#/c/351038
[3] https://review.openstack.org/#/c/384438

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/384438
Committed: https://git.openstack.org/cgit/openstack/charm-hacluster/commit/?id=da39626f0639681097df8b589ba0282c132eaaee
Submitter: Jenkins
Branch: master

commit da39626f0639681097df8b589ba0282c132eaaee
Author: Zhang Hua <email address hidden>
Date: Mon Oct 10 19:34:16 2016 +0800

    Failed to delete ocf resources via delete_resources option

    Now the charm is using 'crm -w -F resource stop <RES>'
    and 'crm -w -F configure delete <RES>' to delete legacy ocf
    resource via delete_resources option, but the 'stop' step will
    fail when ocf file no longer exists (e.g. due to packages no
    longer being available after upgrade). In addition, the daemon
    process may still be running after the upgrade in this case.

    Change-Id: I7fdac7894366a88a154da5a0510acc535ee776d8
    Closes-Bug: #1631858