VMware: launching instance from instance snapshot fails

[Ryan Hsu]

branch: stable/havana

When using the nova VMwareVCDriver, instances launched from instance snapshots are spawning with ERROR.

Steps to Reproduce:
(using Horizon)
1. Launch an instance using a vmdk image
2. Take a snapshot of the instance
3. Launch an instance using the snapshot

Expected result: Instance should boot up successfully
Actual result: Instance boots up with ERROR and following error is found in n-cpu.log:

 Traceback (most recent call last):
   File "/opt/stack/nova/nova/compute/manager.py", line 1407, in _spawn
     block_device_info)
   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 623, in spawn
     admin_password, network_info, block_device_info)
   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 449, in spawn
     uploaded_vmdk_path)
   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 387, in
     self._session._wait_for_task(instance['uuid'], vmdk_copy_task)
   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 900, in
     ret_val = done.wait()
   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116,
     return hubs.get_hub().switch()
   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line
     return self.greenlet.switch()
 NovaException: A specified parameter was not correct.
 fileType

The error seems to occur during the CopyVirtualDisk_Task operation. Full log for context is available here: http://paste.openstack.org/show/50411/

[Gary Kotton (garyk)]

Which image did you use? sparse?
Thanks
Gary

[Gary Kotton (garyk)]

I have confirmed the problem. The issue is when a sparse disk has been snapshotted. Steps to reprodue:

1. glance image-create --name="dos" --disk-format=vmdk --container-format=bare --is-public=true --property vmware_adaptertype="lsiLogic" --property vmware_disktype="sparse" < nostalgia-disk1.vmdk
2. nova boot --image dos --flVOR 1 Y
3. nova image-create Y YY
4. nova boot --image YY --flavor 1 X

The upload to glance should have the relevant flags set.

[Vui Lam (vui)]

Yep, this should be straightforward Maithem. The vmware_disktype property should be preallocated (or unset I guess since it is the default) when uploading to glance.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/55928

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/55929

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/55930

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/55934

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/56113

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/55934
Committed: http://github.com/openstack/nova/commit/11e327f888a6dbde990d75196560083fddecf394
Submitter: Jenkins
Branch: master

commit 11e327f888a6dbde990d75196560083fddecf394
Author: Maithem <email address hidden>
Date: Mon Nov 11 16:08:02 2013 -0800

    VMware: Always upload a snapshot as a preallocated disk

    Currently, a snapshot operation always produces a preallocated
    disk type. When the snapshot is uploaded to glance, the source
    image metadata is uploaded with the new snapshot and then some
    of the metadata properties are updated. The update doesn't
    include the disk_type property, this can be problematic. If
    the original image has a sparse disk type and a snapshot is
    taken, then the snapshot which is a preallocated disk type is
    uploaded to glance with the disk_type property set to sparse,
    which creates a disk type mismatch and causes errors when a vm
    is spawned from that snapshot. This patch addresses the issue
    by correcting the image metadata used when uploading a snapshot
    to glance.

    Also, the image_upload function in vmwareapi/fake.py has been
    modified to address the change. Any function call to
    image_upload (from the unit tests) will assert that the
    metadata for the image to be uploaded must have the disk_type
    value to be preallocated.

    Closes-Bug: 1247296
    Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/74859

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/74859
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9520b8dfc207d0183ef53e84a480270e832d6c70
Submitter: Jenkins
Branch: stable/havana

commit 9520b8dfc207d0183ef53e84a480270e832d6c70
Author: Maithem <email address hidden>
Date: Mon Nov 11 16:08:02 2013 -0800

    VMware: Always upload a snapshot as a preallocated disk

    Currently, a snapshot operation always produces a preallocated
    disk type. When the snapshot is uploaded to glance, the source
    image metadata is uploaded with the new snapshot and then some
    of the metadata properties are updated. The update doesn't
    include the disk_type property, this can be problematic. If
    the original image has a sparse disk type and a snapshot is
    taken, then the snapshot which is a preallocated disk type is
    uploaded to glance with the disk_type property set to sparse,
    which creates a disk type mismatch and causes errors when a vm
    is spawned from that snapshot. This patch addresses the issue
    by correcting the image metadata used when uploading a snapshot
    to glance.

    Also, the image_upload function in vmwareapi/fake.py has been
    modified to address the change. Any function call to
    image_upload (from the unit tests) will assert that the
    metadata for the image to be uploaded must have the disk_type
    value to be preallocated.

    Closes-Bug: 1247296
    (cherry picked from commit 11e327f888a6dbde990d75196560083fddecf394)

    Conflicts:

     nova/tests/virt/vmwareapi/test_vmwareapi.py
    Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765