NetApp: Can't delete a volume in error state

[Sam Morrison]

If a volume isn't created and goes to error state when trying to delete it you get the following error:

2012-12-14 09:55:58 32570 ERROR cinder.openstack.common.rpc.amqp [-] Exception during message handling
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last):
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/amqp.py", line 276, in _process_data
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 206, in delete_volume
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp {'status': 'error_deleting'})
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp self.gen.next()
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 195, in delete_volume
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp self.driver.delete_volume(volume_ref)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/volume/netapp.py", line 500, in delete_volume
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp self._remove_destroy(name, project)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/volume/netapp.py", line 437, in _remove_destroy
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp lun = self._lookup_lun_for_volume(name, project)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/cinder/volume/netapp.py", line 494, in _lookup_lun_for_volume
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp raise exception.VolumeBackendAPIException(data=msg)
2012-12-14 09:55:58 32570 TRACE cinder.openstack.common.rpc.amqp VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: No entry in LUN table for volume volume-0dd5e957-91d8-43a1-843e-75c054e888a5

[John Griffith (john-griffith)]

We must be doing the same sort of testing this week Sam... I have a note to fix this in the SolidFire driver and will need to check the other drivers to make sure this isn't happening.

It seems we assumed if a delete call made it to us that the volume would exist on our back-end systems and we raise. Bad idea, so my approach has bee to remove the raise and instead log an error and move on. It would be nice if we get states implemented and we could use that to determine more appropriately what should be done (raise, log error, nothing etc) but until then I think logging an error is best. For all purposes the volume is deleted as the user requested.

[Sam Morrison (sorrison)]

Yeah I think that would be a good idea, will prevent volumes being impossible to delete.
We're just starting to use cinder and evaluating some netapp kit hence all the bug reports!

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/21195
Committed: http://github.com/openstack/cinder/commit/31b938eae00d5a023bff14567a290eb27de441f9
Submitter: Jenkins
Branch: master

commit 31b938eae00d5a023bff14567a290eb27de441f9
Author: Rushi Agrawal <email address hidden>
Date: Fri Feb 1 23:46:58 2013 +0530

    Fix inability to delete volumes in error state for NetApp driver

    While using NetApp 7-mode storage box to create volumes, if the volume
    is not created on the storage box (possibly due to wrong/missing
    options in cinder.conf file), the volume goes in error state, and when
    one tries to delete the volume, an exception was raised which made
    the volume go in error_deleting state, which is unrecoverable. This
    commit quiesces that exception, resulting in successful removal of the
    volume entry from the volume table.

    Fixes bug 1090167

    Change-Id: I0a25b71ad03e7f7acf58df3952e074ff164b82ff

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/23791

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/24884

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24884
Committed: http://github.com/openstack/cinder/commit/8e702a13ef6c13e33fe84f6347d65cbb8ed0ee3a
Submitter: Jenkins
Branch: stable/folsom

commit 8e702a13ef6c13e33fe84f6347d65cbb8ed0ee3a
Author: Rushi Agrawal <email address hidden>
Date: Wed Mar 6 03:11:55 2013 +0530

    Fix inability to delete volumes in error state for NetApp driver

    While using NetApp 7-mode storage box to create volumes, if the volume
    is not created on the storage box (possibly due to wrong/missing
    options in cinder.conf file), the volume goes in error state, and when
    one tries to delete the volume, an exception was raised which made
    the volume go in error_deleting state, which is unrecoverable. This
    commit quiesces that exception, resulting in successful removal of the
    volume entry from the volume table.

    Fixes bug 1090167

    Change-Id: I0a25b71ad03e7f7acf58df3952e074ff164b82ff