Newton -> Queens FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because cinder-volume pcs disable tasks are not idempotent

[Marius Cornea]

Newton -> Queens FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because openstack-cinder-volume pcs disable tasks are not idempotent

Disabling and deleting the openstack-cinder-volume pacemaker resource tasks are not idempotent:
https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/cinder-volume.yaml#L241-L258

if the upgrade upgrade_steps_playbook.yaml is run multiple times and the openstack-cinder-volume was deleted in the first run then the 2nd time it fails with:

TASK [Disable the openstack-cinder-volume cluster resource] ***********************************************************************************************************************************************************************************
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (5 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (4 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (3 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (2 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (1 retries left).
fatal: [192.168.24.9]: FAILED! => {"attempts": 5, "changed": false, "error": "Error: resource/clone/master/group/bundle 'openstack-cinder-volume' does not exist\n", "failed": true, "msg": "Failed, to set the resource openstack-cinder-volume to the state disable", "output": "", "rc": 1}

We should make the delete tasks idempotent to allow running the playbook multiple times to be able to recover in case of failure.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/537391

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jose Luis Franco (<email address hidden>) on branch: master
Review: https://review.openstack.org/537391
Reason: This patch was included as part of https://review.openstack.org/#/c/539249/8

[Jose Luis Franco (jfrancoa)]

Patch with the solution already merged: https://review.openstack.org/#/c/539249/8
It adds a pacemaker resource check in cinder-backup and cinder-volume templates.