pacemaker_resource_restart.sh hangs on crm_resource --wait

[Jiří Stránský]

When running pacemaker_resource_restart.sh (with fixed resource names using https://review.openstack.org/#/c/296592), the restart hangs after disabling the openstack-core service, on `crm_resource --wait` call which follows this operation.

`crm_resource --wait -VVV` keeps printing this:

  notice: LogActions: Start openstack-cinder-volume (overcloud-controller-2 - blocked)
  notice: LogActions: Start openstack-heat-engine:0 (overcloud-controller-0)
  notice: LogActions: Start openstack-heat-engine:1 (overcloud-controller-1)
  notice: LogActions: Start openstack-heat-engine:2 (overcloud-controller-2)
  notice: LogActions: Start openstack-heat-api-cloudwatch:0 (overcloud-controller-0)
  notice: LogActions: Start openstack-heat-api-cloudwatch:1 (overcloud-controller-1)
  notice: LogActions: Start openstack-heat-api-cloudwatch:2 (overcloud-controller-2)

It's not clear why these resources are trying to start.

There's a ceilometer-notification service still running, i thought that might be the cause why its dependencies are trying to start, but it's odd that e.g. heat-api service is in the same situation wrt dependencies (openstack-core stopped but ceilometer-notification started), but heat-api is *not* trying to start.

Adding the constraints so that all openstack services (incl. ceilometer-notification) depend in some way on openstack-core resource seems to fix the issue.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/304592
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d
Submitter: Jenkins
Branch: master

commit f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d
Author: Jiri Stransky <email address hidden>
Date: Tue Apr 12 13:45:17 2016 +0200

    Make sure openstack services are dependent on openstack-core

    Previously ceilometer-notification, aodh-listener and sahara-engine
    didn't have constraints that would anchor them under openstack-core
    dummy resource. Such constraints are added now. (sahara-engine starting
    after sahara-api, aodh-listener after aodh-evaluator, and
    ceilometer-notification after openstack-core.) Openstack-core ->
    heat-api constraint has been removed because heat-api depends on
    ceilometer-notification, so there's a transitive dependency on
    openstack-core already.

    Change-Id: Ided7321ebbf2c3556726343b4bb466fd8759b43a
    Closes-Bug: #1569444

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/306350

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/306350
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=fae9b282c7b8ef44ca60870b9c00cb4718f87511
Submitter: Jenkins
Branch: stable/mitaka

commit fae9b282c7b8ef44ca60870b9c00cb4718f87511
Author: Jiri Stransky <email address hidden>
Date: Tue Apr 12 13:45:17 2016 +0200

    Make sure openstack services are dependent on openstack-core

    Previously ceilometer-notification, aodh-listener and sahara-engine
    didn't have constraints that would anchor them under openstack-core
    dummy resource. Such constraints are added now. (sahara-engine starting
    after sahara-api, aodh-listener after aodh-evaluator, and
    ceilometer-notification after openstack-core.) Openstack-core ->
    heat-api constraint has been removed because heat-api depends on
    ceilometer-notification, so there's a transitive dependency on
    openstack-core already.

    Change-Id: Ided7321ebbf2c3556726343b4bb466fd8759b43a
    Closes-Bug: #1569444
    (cherry picked from commit f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/tripleo-heat-templates 5.0.0.0b1 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.