Deployment status checking generates tons of messages

[Thomas Herve]

When we create/update a software deployment, we use RPC to talk to another engine for getting its status. The default interval is 1 sec, so we generate lots of messages on the bus to retrieve that status. We should try to reduce this overhead.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/281220
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=a709637a17166cb12e5d94944837bac6451b2851
Submitter: Jenkins
Branch: master

commit a709637a17166cb12e5d94944837bac6451b2851
Author: Thomas Herve <email address hidden>
Date: Tue Feb 16 16:29:51 2016 +0100

    Replace SD RPC polling by long RPC call

    This changes the way SoftwareDeployment updates its internal status.
    Instead of querying the deployment status in a loop using RPC client, it
    makes a single call, which returns once the status has been updated.

    Closes-Bug: #1549219
    Change-Id: I484b7c8cb4a4e71817be6bea764f23b68a39b2d4

[Zane Bitter (zaneb)]

I proposed a revert of the fix because it partially serialises resource creation on non-convergence stacks: https://review.openstack.org/#/c/313122/

At the summit we discussed a possible solution by allowing tasks to indicate to the scheduler how long to wait before polling them again, by doing e.g. "yield 5" (to sleep for 5s) instead of just "yield". This could be useful for many resource types, and of course it could vary dynamically so we could do things like exponential backoffs.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/313148

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/317702

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/313148
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=8f8330573b96ddfb43c5d44f3577c101b34ae299
Submitter: Jenkins
Branch: master

commit 8f8330573b96ddfb43c5d44f3577c101b34ae299
Author: Zane Bitter <email address hidden>
Date: Tue May 17 12:29:12 2016 -0400

    Allow scheduler tasks to request longer wait periods

    This change allows a scheduler co-routine to yield an integer n, such
    that only every nth call to TaskRunner.step() results in the co-routine
    being advanced. We'll be able to use this to selectively slow down
    polling of resources where and when we know it doesn't make sense to
    poll every second.

    Change-Id: Ifc6303524be8e477905546760279027886833617
    Related-Bug: #1549219

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/317702
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=27dd19d72708ccf4c29e37524ec714c38e7f23eb
Submitter: Jenkins
Branch: master

commit 27dd19d72708ccf4c29e37524ec714c38e7f23eb
Author: Zane Bitter <email address hidden>
Date: Tue May 17 12:43:58 2016 -0400

    Allow resources to delay polling

    Provide an easy way for resource types to limit the rate at which they are
    polled, by raising a PollDelay exception from a check_<ACTION>_complete()
    method, without having to maintain their own state.

    Change-Id: I4b3cfb6e07098746e8b46e12fcefd4ee57a85260
    Related-Bug: #1549219

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b1 development milestone.