Failure to power off a VM during delete leads to it going back to Active(None)

[Phil Day]

If libvirt fails to power off a VM during the shutdown called from within do_terminate_instance() it raises exception.InstancePowerOffFailure. However do_terminate_instance doesn’t catch this, and so two (IMO bad) things happen:

i) The instance doesn’t go to an Error (Deleting) state
ii) @reverts_task_state sets task_state to None – putting the instance into Active(None)

This makes the user think the system has just ignored their request, so the repeate the delete, and repeate the delete ….

Proposed changes:
- Catch exception.InstancePowerOffFailure so that the instance goes to Error(Deleting)
- Remove the reverts_task_state decorator from terminate_instance. Delete is a non-reversible operation for the user, and many systems stop billing for instances once the user has indicated that they want to delete the instance (from that point on is the system’s problem to complete the delete as soon as possible. No failure during delete should set the instance back to Active(None)

[Phil Day (philip-day)]

Sorry Aditi - I already have a fix proposed for this: https://review.openstack.org/#/c/58829/

somehow it didn't get linked back to the bug

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/58829
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=63381a15ac3c2e36f5521f8a77108664f89bfab5
Submitter: Jenkins
Branch: master

commit 63381a15ac3c2e36f5521f8a77108664f89bfab5
Author: Phil Day <email address hidden>
Date: Wed Nov 27 17:31:23 2013 +0000

    Failure during termination should always leave state as Error(Deleting)

    Delete is a non-reversible operation for the user, and once the
    user has indicated that they want to delete the instance from that
    point on is the system's problem to complete the delete as soon as
    possible.

    If anything fails during the delete that the system cannot recover
    from then the instance should be left in an Error(Deleting) state.
    Anything else, in particular reverting to an Active(None) state, makes
    it look like the system has ignored the request.

    Currently InstanceTerminationFailure is explicitly caught, but the
    exception does not propagate so the instance_fault wrapper does not
    get a chance to log the failure. Also terminate_instance is wrapped
    by reverts_task_state which resets the state to Active(None)

    This change removes the revert_task_state wrapper and catches all
    exceptions so that unhandled exceptions always leave the instance
    in Error(Deleting)

    Change-Id: I5fb1bbd56035792f566a6e076edfe7a69df006ef
    Closes-Bug: 1254122