cannot delete nova instance if volume is not active

[Eli Qiao]

I booted from an encrypted volume, when try to delete it, it failed

'Device ip-10.238.157.47:3260-iscsi-iqn.2010-10.org.openstack:volume-d167a0ac-fab5-484a-865d-667f1583c2ab-lun-1 is not active.\n'

| fault | {"message": "Unexpected error while running command. |
| | Command: sudo nova-rootwrap /etc/nova/rootwrap.conf cryptsetup luksClose ip-10.238.157.47:3260-iscsi-iqn.2010-10.org.openstack:volume-d167a0ac-fab5-484a-865d-667f1583c2ab-lun-1 |
| | Exit code: 4 |
| | Stdout: u'' |
| | Stderr: u'Dev", "code": 500, "details": " File \"/opt/stack/nova/nova/compute/manager.py\", line 351, in decorated_function |
| | return function(self, context, *args, **kwargs) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2370, in terminate_instance |
| | do_terminate_instance(instance, bdms) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py\", line 252, in inner |
| | return f(*args, **kwargs) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2368, in do_terminate_instance |
| | self._set_instance_error_state(context, instance) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 119, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2358, in do_terminate_instance |
| | self._delete_instance(context, instance, bdms, quotas) |
| | File \"/opt/stack/nova/nova/hooks.py\", line 149, in inner |
| | rv = f(*args, **kwargs) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2337, in _delete_instance |
| | quotas.rollback() |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 119, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2307, in _delete_instance |
| | self._shutdown_instance(context, instance, bdms) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2234, in _shutdown_instance |
| | requested_networks) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py\", line 119, in __exit__ |
| | six.reraise(self.type_, self.value, self.tb) |
| | File \"/opt/stack/nova/nova/compute/manager.py\", line 2223, in _shutdown_instance |
| | block_device_info) |
| | File \"/opt/stack/nova/nova/virt/libvirt/driver.py\", line 768, in destroy |
| | destroy_disks, migrate_data) |
| | File \"/opt/stack/nova/nova/virt/libvirt/driver.py\", line 844, in cleanup |
| | encryptor.detach_volume(**encryption) |
| | File \"/opt/stack/nova/nova/volume/encryptors/cryptsetup.py\", line 101, in detach_volume |
| | self._close_volume(**kwargs) |
| | File \"/opt/stack/nova/nova/volume/encryptors/luks.py\", line 128, in _close_volume |
| | attempts=3) |
| | File \"/opt/stack/nova/nova/utils.py\", line 229, in execute |
| | return processutils.execute(*cmd, **kwargs) |
| | File \"/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py\", line 275, in execute |
| | cmd=sanitized_cmd) |
| | ", "created": "2015-08-06T03:30:52Z"} |

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/215252

[Matt Riedemann (mriedem)]

I don't think this is a regression in liberty, so marking as liberty-backport-potential so it's not blocking the liberty rc1.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/215252
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=11e20dd6f91d2417e45b44d8c9008b07ebdcec28
Submitter: Jenkins
Branch: master

commit 11e20dd6f91d2417e45b44d8c9008b07ebdcec28
Author: Eli Qiao <email address hidden>
Date: Tue Oct 13 14:08:31 2015 +0800

    Ignore errorcode=4 when executing `cryptsetup remove` command

    If an attached encrypted volume is failed to detach from the instance
    when deleting that instance, user can not delete that instance at all.

    This patch adds 4 in check_exit_code when executing `cryptsetup remove`
    command to eat that exception.

    PS: exit_code = 4 indicate ENODEV error which means no device(also includes
    the crypt device inactive).

    Closes-Bug: #1482066
    Change-Id: I12e2a52068850528a4bd68486344b74eb9b82c88

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/nova 13.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/288480

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/288480
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=20c3335478de9495c55ce63a3a21fab97de1642f
Submitter: Jenkins
Branch: stable/liberty

commit 20c3335478de9495c55ce63a3a21fab97de1642f
Author: Eli Qiao <email address hidden>
Date: Tue Oct 13 14:08:31 2015 +0800

    Ignore errorcode=4 when executing `cryptsetup remove` command

    If an attached encrypted volume is failed to detach from the instance
    when deleting that instance, user can not delete that instance at all.

    This patch adds 4 in check_exit_code when executing `cryptsetup remove`
    command to eat that exception.

    PS: exit_code = 4 indicate ENODEV error which means no device(also includes
    the crypt device inactive).

    Closes-Bug: #1482066
    Change-Id: I12e2a52068850528a4bd68486344b74eb9b82c88
    (cherry picked from commit 11e20dd6f91d2417e45b44d8c9008b07ebdcec28)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/nova 12.0.3 release.