VMware: Detaching a volume from an instance also deletes the volume's backing vmdk (ESXDriver only)

Bug #1241350 reported by Subbu
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Guangya Liu (Jay Lau)
Havana
Fix Released
High
Gary Kotton

Bug Description

I found that when I run:

% nova volume-detach my_instance c54ad11f-4e51-41a0-97db-7e551776db59

where the volume with given id is currently attached to my running instance named my_instance, the operation completes successfully. Nevertheless a subsequent attach of the same volume again will fail. So:

% nova volume-attach my_instance c54ad11f-4e51-41a0-97db-7e551776db59 /dev/sdb

fails with the error that the volume's vmdk file is not found.

Cause:

During volume detach a delete_virtual_disk_spec is used to remove the device from the running instance. This spec also "destroy"s the underlying vmdk file. The offending line is : https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/vm_util.py#L471

Possible fix:
The fileOperation field of the device config during this reconfigure operation should be left unset. We should continue setting device_config.operation field to "remove". This will remove the device from the VM without deleting the underlying vmdk backing.

Tags: vmware
Changed in nova:
assignee: nobody → Jay Lau (jay-lau-513)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/52645

Changed in nova:
status: New → In Progress
Gary Kotton (garyk)
tags: added: havana-backport-potential havana-rc-potential
Changed in nova:
importance: Undecided → Critical
importance: Critical → High
Tracy Jones (tjones-i)
Changed in openstack-vmwareapi-team:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Jay Lau (jay-lau-513)
Revision history for this message
Guangya Liu (Jay Lau) (jay-lau-513) wrote : Re: VMware: Detaching a volume from an instance also deletes the volume's backing vmdk

Hi Tracy, just curious what can I do for "VMwareAPI-Team"? ;-) Can you please show more details? Thanks.

Revision history for this message
Tracy Jones (tjones-i) wrote :

Jay - this is just a way where we track bugs that are important for our customers. think of this as kind of a tag.

Revision history for this message
Subbu (subramanian-neelakantan) wrote :

I found that this problem occurs only when using the VMwareESXDriver and not with the VMwareVCDriver.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/52645
Committed: http://github.com/openstack/nova/commit/0405d7b937fdeece46efc1ba90171163f3286a56
Submitter: Jenkins
Branch: master

commit 0405d7b937fdeece46efc1ba90171163f3286a56
Author: Jay Lau <email address hidden>
Date: Fri Oct 18 22:38:13 2013 +0800

    VMware: Detach volume should not delete vmdk

    During volume detach a delete_virtual_disk_spec is used to remove
    the device from the running instance. This spec also "destroy" the
    underlying vmdk file, this is not right, we should not delete vmdk
    file when detach it from a VM instance.

    The fix was remove the fileOperation field when detach vmdk volume
    so as to make sure the volume will not be destroyed; but for iscsi
    we still need to destroy the volume.

    Also I changed the function name from delete_virtual_disk_spec to
    detach_virtual_disk_spec which is more accurate.

    Change-Id: Ibd218d6a8cfeede4f5ca74f28bc0c3d0c185bb14
    Closes-Bug: #1241350

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/56746

Revision history for this message
dan wendlandt (danwent) wrote :

Removing vmwareapi team tag, as this is not relevant for the VCDriver, only relevant for the ESXDriver.

summary: VMware: Detaching a volume from an instance also deletes the volume's
- backing vmdk
+ backing vmdk (ESXDriver only)
no longer affects: openstack-vmwareapi-team
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/havana)

Reviewed: https://review.openstack.org/56746
Committed: http://github.com/openstack/nova/commit/e69310b27bf1c89ac9b971720b62f1cc8268de3c
Submitter: Jenkins
Branch: stable/havana

commit e69310b27bf1c89ac9b971720b62f1cc8268de3c
Author: Jay Lau <email address hidden>
Date: Fri Oct 18 22:38:13 2013 +0800

    VMware: Detach volume should not delete vmdk

    During volume detach a delete_virtual_disk_spec is used to remove
    the device from the running instance. This spec also "destroy" the
    underlying vmdk file, this is not right, we should not delete vmdk
    file when detach it from a VM instance.

    The fix was remove the fileOperation field when detach vmdk volume
    so as to make sure the volume will not be destroyed; but for iscsi
    we still need to destroy the volume.

    Also I changed the function name from delete_virtual_disk_spec to
    detach_virtual_disk_spec which is more accurate.

    Change-Id: Ibd218d6a8cfeede4f5ca74f28bc0c3d0c185bb14
    Closes-Bug: #1241350
    (cherry picked from commit 0405d7b937fdeece46efc1ba90171163f3286a56)

tags: added: in-stable-havana
Changed in nova:
milestone: none → icehouse-1
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Alan Pevec (apevec)
tags: removed: havana-backport-potential havana-rc-potential in-stable-havana
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.