pacemaker_resource_restart.sh hangs on crm_resource --wait

Bug #1569444 reported by Jiří Stránský
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Jiří Stránský
Mitaka
Fix Released
Undecided
Unassigned

Bug Description

When running pacemaker_resource_restart.sh (with fixed resource names using https://review.openstack.org/#/c/296592), the restart hangs after disabling the openstack-core service, on `crm_resource --wait` call which follows this operation.

`crm_resource --wait -VVV` keeps printing this:

  notice: LogActions: Start openstack-cinder-volume (overcloud-controller-2 - blocked)
  notice: LogActions: Start openstack-heat-engine:0 (overcloud-controller-0)
  notice: LogActions: Start openstack-heat-engine:1 (overcloud-controller-1)
  notice: LogActions: Start openstack-heat-engine:2 (overcloud-controller-2)
  notice: LogActions: Start openstack-heat-api-cloudwatch:0 (overcloud-controller-0)
  notice: LogActions: Start openstack-heat-api-cloudwatch:1 (overcloud-controller-1)
  notice: LogActions: Start openstack-heat-api-cloudwatch:2 (overcloud-controller-2)

It's not clear why these resources are trying to start.

There's a ceilometer-notification service still running, i thought that might be the cause why its dependencies are trying to start, but it's odd that e.g. heat-api service is in the same situation wrt dependencies (openstack-core stopped but ceilometer-notification started), but heat-api is *not* trying to start.

Adding the constraints so that all openstack services (incl. ceilometer-notification) depend in some way on openstack-core resource seems to fix the issue.

Changed in tripleo:
assignee: nobody → Jiří Stránský (jistr)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.openstack.org/304592
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d
Submitter: Jenkins
Branch: master

commit f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d
Author: Jiri Stransky <email address hidden>
Date: Tue Apr 12 13:45:17 2016 +0200

    Make sure openstack services are dependent on openstack-core

    Previously ceilometer-notification, aodh-listener and sahara-engine
    didn't have constraints that would anchor them under openstack-core
    dummy resource. Such constraints are added now. (sahara-engine starting
    after sahara-api, aodh-listener after aodh-evaluator, and
    ceilometer-notification after openstack-core.) Openstack-core ->
    heat-api constraint has been removed because heat-api depends on
    ceilometer-notification, so there's a transitive dependency on
    openstack-core already.

    Change-Id: Ided7321ebbf2c3556726343b4bb466fd8759b43a
    Closes-Bug: #1569444

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/306350

tags: added: mitaka-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/mitaka)

Reviewed: https://review.openstack.org/306350
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=fae9b282c7b8ef44ca60870b9c00cb4718f87511
Submitter: Jenkins
Branch: stable/mitaka

commit fae9b282c7b8ef44ca60870b9c00cb4718f87511
Author: Jiri Stransky <email address hidden>
Date: Tue Apr 12 13:45:17 2016 +0200

    Make sure openstack services are dependent on openstack-core

    Previously ceilometer-notification, aodh-listener and sahara-engine
    didn't have constraints that would anchor them under openstack-core
    dummy resource. Such constraints are added now. (sahara-engine starting
    after sahara-api, aodh-listener after aodh-evaluator, and
    ceilometer-notification after openstack-core.) Openstack-core ->
    heat-api constraint has been removed because heat-api depends on
    ceilometer-notification, so there's a transitive dependency on
    openstack-core already.

    Change-Id: Ided7321ebbf2c3556726343b4bb466fd8759b43a
    Closes-Bug: #1569444
    (cherry picked from commit f5d96bb41bb7d19de8649b428fbac8c94f0dfd9d)

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/tripleo-heat-templates 5.0.0.0b1

This issue was fixed in the openstack/tripleo-heat-templates 5.0.0.0b1 development milestone.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/tripleo-heat-templates 2.1.0

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

This issue was fixed in the openstack/tripleo-heat-templates 2.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.