Ensure create/update greenthreads are always cancelled on delete

[Steven Hardy]

Related to bug 1096150, which fixed one specific failure-mode where a ThreadGroup.stop() failed to kill the running greenthread - we have further work to do to ensure this is robust, since there are still multiple generic "except Exception" clauses in the code-path of the create/update greenthreads, so we are susceptible to random breakage if a stack is deleted and we happen to have a greenthread operation in progress (create/update stack, periodic watcher task)

The real problem here is eventlet, as described in this oslo bug:

https://bugs.launchpad.net/oslo/+bug/1097203

However, if a solution to this can't be found at the oslo/ThreadGroup level, then we need to remove all generic exception handling in our code, and ensure all try/except blocks are fully qualified (such that the except *can't* mistakenly catch the GreenletExit exception allowing the thread to continue running after a greenthread.kill())

The main areas I spotted which we definitely need to fix:
- Resource.update() : Needs the same fix as applied for bug 1096150
- All resource.py DB manipulation methods (_store_or_update, _add_event, _store, resource_id_set), otherwise a stack deletion could be caught and mistakenly logged as a DB error.
- ditto WatchRule.load()

Probably some others, further investigation required.

[Zane Bitter (zaneb)]

I have a patch under review in Oslo to monkey-patch greenlet to fix this.

[Steven Dake (sdake)]

Zane,

Is this fixed by this review?

https://review.openstack.org/#/c/20026/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24293

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24293
Committed: http://github.com/openstack/heat/commit/8e4938525793fb601c7372bf9bcb69fe33ebecb3
Submitter: Jenkins
Branch: master

commit 8e4938525793fb601c7372bf9bcb69fe33ebecb3
Author: Zane Bitter <email address hidden>
Date: Wed Mar 13 09:28:14 2013 +0100

    Update dependency versions to match oslo-incubator

    Fixes bug 1097843

    Change-Id: I3bd56261ed9e3640adb07965d4394d99db45e0b5