pcmk_remote is not ordered after wait-for-settle on rhel8

[Michele Baldessari]

We currently have the following ordering constraints collectors:
  Exec<| title == 'wait-for-settle' |> -> Pcmk_constraint<||>
  Exec<| title == 'wait-for-settle' |> -> Pcmk_resource<||>
  Exec<| title == 'wait-for-settle' |> -> Pcmk_property<||>
  Exec<| title == 'wait-for-settle' |> -> Pcmk_bundle<||>

The problem here is that on rhel8 remotes are a dedicated puppet resource 'pcmk_remote'. From manifests/resource/remote.pp:
  if $::pacemaker::params::pcs_010 {
    pcmk_remote { $name:
      ensure => $ensure,
...
  } else {
    pcmk_resource { $name:
      ensure => $ensure,

This means that on rhel8 we do not have the guarantee that remotes will be created after the cluster has settled. This has been observed in a real-life scenario (the wait-for-settle took place *after* the error):
Mar 12 01:51:05 controller-0.redhat.local puppet-user[27085]: Error: /Stage[main]/Tripleo::Profile::Base::Pacemaker/Pacemaker::Resource::Remote[compute-0]/Pcmk_remote[compute-0]: Could not evaluate: backup_cib: Running: pcs cluster cib /var/lib/pacemaker/cib/puppet-cib-backup20200312-27084-1ewlz0x failed with code: 1 -> Error: unable to get cib

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/712623

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/712623
Committed: https://git.openstack.org/cgit/openstack/puppet-pacemaker/commit/?id=e1f66f226a696f02d58784832235d2663c39887e
Submitter: Zuul
Branch: master

commit e1f66f226a696f02d58784832235d2663c39887e
Author: Michele Baldessari <email address hidden>
Date: Thu Mar 12 08:55:09 2020 +0100

    order pcmk_remote after Exec['wait-for-settle']

    We currently have the following ordering constraints collectors:
      Exec<| title == 'wait-for-settle' |> -> Pcmk_constraint<||>
      Exec<| title == 'wait-for-settle' |> -> Pcmk_resource<||>
      Exec<| title == 'wait-for-settle' |> -> Pcmk_property<||>
      Exec<| title == 'wait-for-settle' |> -> Pcmk_bundle<||>

    The problem here is that on rhel8 remotes are a dedicated puppet resource
    'pcmk_remote'. From manifests/resource/remote.pp:
      if $::pacemaker::params::pcs_010 {
        pcmk_remote { $name:
          ensure => $ensure,
    ...
      } else {
        pcmk_resource { $name:
          ensure => $ensure,

    This means that on rhel8 we do not have the guarantee that remotes will be
    created after the cluster has settled. This has been observed in a real-life
    scenario (the wait-for-settle took place *after* the error):

    Mar 12 01:51:05 controller-0.redhat.local puppet-user[27085]: Error: /Stage[main]/Tripleo::Profile::Base::Pacemaker/Pacemaker::Resource::Remote[compute-0]/Pcmk_remote[compute-0]: Could not evaluate: backup_cib: Running: pcs cluster cib /var/lib/pacemaker/cib/puppet-cib-backup20200312-27084-1ewlz0x failed with code: 1 -> Error: unable to get cib

    With this change we did not observe the above error any longer.

    Co-Authored-By: Luca Miccini <email address hidden>
    Closes-Bug: #1867105

    Change-Id: Id46d22d9a0e4d092c45aed1e90d20669c6808d64