OpenStack Compute (nova)

[ietd] can not delete volume sucessfully

Bug #1055401 reported by MotoKen on 2012-09-24

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Cinder	Fix Released	High	MotoKen	Cinder 2012.2 "folsom"
	OpenStack Compute (nova)	Fix Released	High	MotoKen	OpenStack Compute (nova) 2012.2 "folsom"

Bug Description

trackback:

2012-09-24 17:36:31 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 275, in _process_data
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/distpackages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/volume/manager.py", line 209, in delete_volume
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp {'status': 'error_deleting'})
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp self.gen.next()
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/volume/manager.py", line 198, in delete_volume
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp self.driver.delete_volume(volume_ref)
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/volume/driver.py", line 202, in delete_volume
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp self._delete_volume(volume, volume['size'])
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/volume/driver.py", line 154, in _delete_volume
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp run_as_root=True)
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/volume/driver.py", line 97, in _try_execute
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp self._execute(*command, **kwargs)
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 206, in execute
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd))
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command.
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp Command: sudo nova-rootwrap /etc/nova/rootwrap.conf dmsetup remove -f /dev/mapper/nova--volumes-volume--b728f1b8--1659--4344--b2e2--ad9c4e9ba338
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp Exit code: 1
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp Stdout: ''
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp Stderr: 'device-mapper: remove ioctl failed: Device or resource busy\nCommand failed\n'
2012-09-24 17:36:31 TRACE nova.openstack.common.rpc.amqp

Thierry Carrez (ttx) on 2012-09-24

tags:

added: folsom-rc-potential

Revision history for this message

John Griffith (john-griffith) wrote on 2012-09-25:

Motoken,

Could you give me some more details about your setup here, specifically OS?

I was considering putting an lsof -t call in before the remove from dev/mapper device and doing a kill -9 if the device was *open*. I'd rather not do this however, and get a clear understanding of root cause and who/what has the device open. If you can easily reproduce, or share how to reproduce maybe we can get to the bottom of it.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/13616

Changed in nova:
assignee:	nobody → MotoKen (motokentsai)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/13617

Changed in cinder:
assignee:	nobody → MotoKen (motokentsai)
status:	New → In Progress

Thierry Carrez (ttx) on 2012-09-25

Changed in cinder:
importance:	Undecided → High
Changed in nova:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix merged to cinder (master)

Reviewed: https://review.openstack.org/13617
Committed: http://github.com/openstack/cinder/commit/5b61536fdf17332f48af4b422fe23c2e153934a1
Submitter: Jenkins
Branch: master

commit 5b61536fdf17332f48af4b422fe23c2e153934a1
Author: MotoKen <email address hidden>
Date: Tue Sep 25 13:08:34 2012 +0800

Fixes remove_export for IetAdm

* Corrects indentation.
* Uses specific iqn as keyword argument.

Fixes bug 1055401

Change-Id: Ie99dd8a34924f0d7f447cf41a305353659a51d11

Changed in cinder:
status:	In Progress → Fix Committed

John Griffith (john-griffith) on 2012-09-25

Changed in cinder:
milestone:	none → folsom-rc3

Vish Ishaya (vishvananda) on 2012-09-25

Changed in nova:
milestone:	none → folsom-rc3
tags:	removed: folsom-rc-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to cinder (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/13672

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/13675

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-25: Fix merged to cinder (milestone-proposed)

Reviewed: https://review.openstack.org/13672
Committed: http://github.com/openstack/cinder/commit/615f336cc0359a83dd72bc8baf5d61bd69404959
Submitter: Jenkins
Branch: milestone-proposed

commit 615f336cc0359a83dd72bc8baf5d61bd69404959
Author: MotoKen <email address hidden>
Date: Tue Sep 25 13:08:34 2012 +0800

Fixes remove_export for IetAdm

* Corrects indentation.
* Uses specific iqn as keyword argument.

Fixes bug 1055401

Change-Id: Ie99dd8a34924f0d7f447cf41a305353659a51d11
(cherry picked from commit 5b61536fdf17332f48af4b422fe23c2e153934a1)

Changed in cinder:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-26: Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/13675
Committed: http://github.com/openstack/nova/commit/1fae8cea1d745f7756e77916b23c52bf4f05e87b
Submitter: Jenkins
Branch: milestone-proposed

commit 1fae8cea1d745f7756e77916b23c52bf4f05e87b
Author: MotoKen <email address hidden>
Date: Tue Sep 25 12:43:26 2012 +0800

Fixes remove_export for IetAdm

* Corrects indentation.
* Uses specific iqn as keyword argument.

Fixes bug 1055401

Change-Id: I1267be37a7e7adcd2a18ffb719326c5f68296523
(cherry picked from commit 8ab365c5a9302d9770807e9ee8a6ed4100304298)

Changed in nova:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-09-26: Fix merged to nova (master)

Reviewed: https://review.openstack.org/13616
Committed: http://github.com/openstack/nova/commit/8ab365c5a9302d9770807e9ee8a6ed4100304298
Submitter: Jenkins
Branch: master

commit 8ab365c5a9302d9770807e9ee8a6ed4100304298
Author: MotoKen <email address hidden>
Date: Tue Sep 25 12:43:26 2012 +0800

Fixes remove_export for IetAdm

* Corrects indentation.
* Uses specific iqn as keyword argument.

Fixes bug 1055401

Change-Id: I1267be37a7e7adcd2a18ffb719326c5f68296523

Thierry Carrez (ttx) on 2012-09-26

Changed in nova:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2012-09-27

Changed in cinder:
milestone:	folsom-rc3 → 2012.2

Thierry Carrez (ttx) on 2012-09-27

Changed in nova:
milestone:	folsom-rc3 → 2012.2

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.