Failed VM's not set to error state on exception

[moorryan]

We've identified that when an instance build fails because it has exceeded the number of retries (i.e. been tried on 3 different hosts) the upstream logic in the scheduler does not correctly set the VM state to ERROR.

There is an exception handler for NoValidHost in manager.run_instance(), but it relies on request_spec[instance_uuids] to determine which instances to put into the Error State, and schedule_run_instance removes this value (as its normally about to split the request up into several separate requests).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/29780

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/29780
Committed: http://github.com/openstack/nova/commit/aefc28dd481354edd0f3b5aec18db006680b2ffe
Submitter: Jenkins
Branch: master

commit aefc28dd481354edd0f3b5aec18db006680b2ffe
Author: Ryan Moore <email address hidden>
Date: Mon May 20 14:56:55 2013 +0100

    set ERROR state when scheduler hits max attempts

    Presently when scheduler raises NoValidHost due to max attempts
    being reached, the instance remains in a build state.
    Exception handler for NoValidHost in manager.run_instance() needs
    request_spec[instance_uuids] to know which host to put into an
    error state in _set_vm_state_and_notify().
    schedule_run_instances() was popping instance_uuids from the
    request_spec prior to a call to _schedule().
    Changed pop of instance_uuids prior to call to _schedule() to be a get.
    Added pop of instance_uuids to beneath call to _schedule() as
    individual creates do not need them.

    Change-Id: I9654820e01d5611763e9e673f15f46b947d09e6d
    Fixes: bug #1182056

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/30892

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30892
Committed: http://github.com/openstack/nova/commit/7726dae1c3720477191cfdb239fdd2c5d0952285
Submitter: Jenkins
Branch: stable/grizzly

commit 7726dae1c3720477191cfdb239fdd2c5d0952285
Author: Ryan Moore <email address hidden>
Date: Mon May 20 14:56:55 2013 +0100

    set ERROR state when scheduler hits max attempts

    Presently when scheduler raises NoValidHost due to max attempts
    being reached, the instance remains in a build state.
    Exception handler for NoValidHost in manager.run_instance() needs
    request_spec[instance_uuids] to know which host to put into an
    error state in _set_vm_state_and_notify().
    schedule_run_instances() was popping instance_uuids from the
    request_spec prior to a call to _schedule().
    Changed pop of instance_uuids prior to call to _schedule() to be a get.
    Added pop of instance_uuids to beneath call to _schedule() as
    individual creates do not need them.

    Conflicts:
     nova/scheduler/filter_scheduler.py

    Change-Id: I9654820e01d5611763e9e673f15f46b947d09e6d
    Fixes: bug #1182056
    (cherry picked from commit aefc28dd481354edd0f3b5aec18db006680b2ffe)