Comment 2 for bug 1875890

Ok so here is the root cause:
puppet-pacemaker-0.7.3-0.20190930085227.447cef0.el7.noarch has a an older broken match regex to determine when it is running on rhel8, namely from https://github.com/openstack/puppet-pacemaker/blob/447cef0ad8891e89495fef7a9be5e2152b07edfa/manifests/params.pp we see it has:
      if $::operatingsystemrelease =~ /8\..*$/ {
        $pcs_010 = true
      } else {
        $pcs_010 = false
      }

That code will match on rhel 7.8 and so we basically kick off the codepaths that meant for Centos 8 and pcs 0.10 as opposed to pcs 0.9x and centos 7.

What I am not 100% sure is why we did not see an error message in the logs eventually and instead we timed out, because once I ran puppet --debug by hand on the env Panda gave me I saw the errors and it failed after 10 tries.

In any case the fix here is simply to rebase puppet-pacemaker in our releases and be done with it. This has been fixed in master and we do not really use branches for puppet-pacemaker.

I now tested a queens deploy on rhel 7.8 with puppet-pacemaker from master and it worked okay, so I think this should be pretty safe.

I will now test a deploy of queens+7.8 with stock "old" puppet-pacemaker and then redeploy against that with an updated puppet-pacemaker to make sure there are no surprises. If that is okay I will post a review to rdoinfo to move the pins to a puppet-pacemaker that has the operating system fix.