Can't force-delete instance with task_state not 'None'

[Rajesh Tailor]

Problem Description:

When user tries to force-delete instance which is not in task_state 'None', throws 500 error on console, as below:

ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)

Steps to reproduce:
1) Create instance.
$ nova boot --flavor <flavor-id> --image <image-id> <instance-name>

2) To change the instance task_state to anything other than 'None', I tried to resize the instance.
$ nova resize <instance-id> <flavor-id>

3) Try to force-delete instance, just after above step, so that instance task state is anything in (resize_prep, resize_migrating, resize_migrated, resize_finish), but not 'None'.
$ nova force-delete <instance-id>

4) User gets below error on console:

ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)

Actual result:
User gets 500 ClientException Error.

Expected result:
The instance should be deleted without any error.

traceback from nova-api logs:

DEBUG oslo_concurrency.lockutils [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Lock "fbe6eec8-be64-4473-8434-1d795e7ca5fb" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.0
00s {{(pid=24369) inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285}}
ERROR nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Unexpected exception in API method: InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
ERROR nova.api.openstack.wsgi Traceback (most recent call last):
ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 803, in wrapped
ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 61, in _force_delete
ERROR nova.api.openstack.wsgi self.compute_api.force_delete(context, instance)
ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 201, in inner
ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 141, in inner
ERROR nova.api.openstack.wsgi method=f.__name__)
ERROR nova.api.openstack.wsgi InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
ERROR nova.api.openstack.wsgi
INFO nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'>
DEBUG nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

NOTE: If user tries delete api instead of force-delete api for instance deletion, in that case instance is deleted and no error is thrown.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/530879

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/530879
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0d2031a5c408ef5919387c680eeaac68709a0f6c
Submitter: Zuul
Branch: master

commit 0d2031a5c408ef5919387c680eeaac68709a0f6c
Author: Rajesh Tailor <email address hidden>
Date: Wed Jan 3 13:31:38 2018 +0530

    Allow force-delete even if task_state is not None

    As of now, if vm task_state is not 'None', and user tries
    to force-delete instance, then he gets HTTP 500 Error
    and instance deletion doesn't progress.

    The same is not the case, when user tries with delete api
    instead of force-delete api, even if vm task_state is not 'None'.

    Fixed the issue by allowing force-delete to delete instance
    in task_state other than None.

    Change-Id: Ida1a9d8761cec9585f031ec25e5692b8bb55661e
    Closes-Bug: #1741000

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/543343

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/543345

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/543343
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=49685c58454984a3e24225f8d88902f749f4b9c0
Submitter: Zuul
Branch: stable/queens

commit 49685c58454984a3e24225f8d88902f749f4b9c0
Author: Rajesh Tailor <email address hidden>
Date: Wed Jan 3 13:31:38 2018 +0530

    Allow force-delete even if task_state is not None

    As of now, if vm task_state is not 'None', and user tries
    to force-delete instance, then he gets HTTP 500 Error
    and instance deletion doesn't progress.

    The same is not the case, when user tries with delete api
    instead of force-delete api, even if vm task_state is not 'None'.

    Fixed the issue by allowing force-delete to delete instance
    in task_state other than None.

    Change-Id: Ida1a9d8761cec9585f031ec25e5692b8bb55661e
    Closes-Bug: #1741000
    (cherry picked from commit 0d2031a5c408ef5919387c680eeaac68709a0f6c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/543345
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=29b3eada891d9599eb3c284af2174868cb11a4fd
Submitter: Zuul
Branch: stable/pike

commit 29b3eada891d9599eb3c284af2174868cb11a4fd
Author: Rajesh Tailor <email address hidden>
Date: Wed Jan 3 13:31:38 2018 +0530

    Allow force-delete even if task_state is not None

    As of now, if vm task_state is not 'None', and user tries
    to force-delete instance, then he gets HTTP 500 Error
    and instance deletion doesn't progress.

    The same is not the case, when user tries with delete api
    instead of force-delete api, even if vm task_state is not 'None'.

    Fixed the issue by allowing force-delete to delete instance
    in task_state other than None.

    Change-Id: Ida1a9d8761cec9585f031ec25e5692b8bb55661e
    Closes-Bug: #1741000
    (cherry picked from commit 0d2031a5c408ef5919387c680eeaac68709a0f6c)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 17.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 18.0.0.0b1 development milestone.