VMware: volume create ignores adapter_type

[Eric Brown]

I believe this issue is part of the root cause for this nova bug: https://bugs.launchpad.net/nova/+bug/1255317

Basically, when creating a volume from a vmdk image, the adapter_type is ignored and it defaults to LSI logic.
https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/vmware/volumeops.py#L368
https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/vmware/volumeops.py#L327

So cinder creates a volume with LSI SCSI controller and IDE disk. And eventually, when I try to boot from a volume in Nova, I get "No operating system found". I don't think you can boot an IDE disk using SCSI adapter.

However, I am able to boot from an image with IDE adapter. So seems the volume step is causing an issue.

My recreate steps:
1) glance image-create --name cirros-sparse --is-public=True --container-format=bare --disk-format=vmdk --property vmware_disktype="sparse" < cirros-0.3.0-i386-disk.vmdk
2) cinder create --name sparse-ide --image-id cd0a8dea-2e2f-43be-97ab-85e21375b757 1
3) Observe in vCenter that the newly created volume has adapter set as LSI logic.

My VMDK image metadata:

KDMV?9??

# Disk DescriptorFile
version=1
CID=5269c92d
parentCID=ffffffff
createType="monolithicSparse"

# Extent description
RW 80325 SPARSE "disk-vmdk.vmdk"
ericwb@ericwb-virtual-machine:~$ head -n 20disk-vmdk.vmdk
head: 20disk-vmdk.vmdk: invalid number of lines
ericwb@ericwb-virtual-machine:~$ head -n 20 disk-vmdk.vmdk
KDMV?9??

# Disk DescriptorFile
version=1
CID=5269c92d
parentCID=ffffffff
createType="monolithicSparse"

# Extent description
RW 80325 SPARSE "disk-vmdk.vmdk"

# The Disk Data Base
#DDB

ddb.virtualHWVersion = "4"
ddb.geometry.cylinders = "79"
ddb.geometry.heads = "16"
ddb.geometry.sectors = "63"
ddb.adapterType = "ide"

[Eric Brown (ericwb)]

Its not just the adapter_type affecting the boot of the volume. It appears the sparse tag is also causing an issue. I know because I tweaked my cinder devstack to acknowledge IDE. So handling sparse would also have to be part of this fix.

From my testing, you should be able to create a volume in Openstack and manually start the VM for the volume in vCenter without error. This is not true of this sparse-ide image. https://www.dropbox.com/s/793n37blb4ra3t6/disk-vmdk.vmdk

[Vipin Balachandran (vbala)]

Related bugs:
https://bugs.launchpad.net/cinder/+bug/1287176
https://bugs.launchpad.net/cinder/+bug/1287185

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/98057

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/101518

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/98057
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f9402ffe9fbb1ad00fa283a5bf114dc70d2de23f
Submitter: Jenkins
Branch: master

commit f9402ffe9fbb1ad00fa283a5bf114dc70d2de23f
Author: Vipin Balachandran <email address hidden>
Date: Thu Jun 5 12:25:53 2014 +0530

    VMware: Optional create backing parameters

    The current create backing methods do not support specifying an adapter
    type for the backing VM. These methods always create a backing VM with
    a single disk and LSI logic adapter. This change adds optional parameters
    to create backing methods so that a backing VM can be created without a
    disk or with a specific adapter type.

    Partial-Bug: #1284284
    Partial-Bug: #1287185
    Partial-Bug: #1287176
    Change-Id: Ifff64eb2be4af1c4218e810366e25dbecdc5847f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/101518
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=b07c1a759c8a0cdaaaaaf3e6f1ad7402bf631a44
Submitter: Jenkins
Branch: master

commit b07c1a759c8a0cdaaaaaf3e6f1ad7402bf631a44
Author: Vipin Balachandran <email address hidden>
Date: Thu Jun 5 16:21:27 2014 +0530

    VMware:Fix params for copy-image-to-volume

    While creating a volume from stream-optimized image, the adapter type
    in the image meta-data and the profile in the volume type extra spec
    are ignored. This change fix those missing parameters.

    Partial-Bug: #1284284
    Closes-Bug: #1332482
    Change-Id: I46c4755989fe61e0e654ca6daa3c3af0655071f0

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/106038

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/106113

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/106334

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/106038
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=97fbe770bc4dc97218e170103bfe16a1ad632976
Submitter: Jenkins
Branch: master

commit 97fbe770bc4dc97218e170103bfe16a1ad632976
Author: Vipin Balachandran <email address hidden>
Date: Sun Jun 29 18:57:00 2014 +0530

    VMware:Support for create disk and disk descriptor

    This change adds utility methods for creating a virtual disk and virtual
    disk descriptor. Currently volume creation from preallocated image doesn't
    honour the disk type in the volume extra spec and adapter type in the image
    meta-data. The workflow to address these issues require support for the
    above mentioned methods.

    Change-Id: If8ec1fd6c60d26552e5fb58b8bb16a8cc87d9d28
    Partial-Bug: #1284284
    Partial-Bug: #1287185

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/106113
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=15a93312641bb6121d20c7749481104a1ab34bd0
Submitter: Jenkins
Branch: master

commit 15a93312641bb6121d20c7749481104a1ab34bd0
Author: Vipin Balachandran <email address hidden>
Date: Sun Jun 29 19:00:04 2014 +0530

    VMware:Support for attaching disk to backing

    This change adds support for attaching a virtual disk to an existing
    backing VM. Currently volume creation from preallocated/sparse image doesn't
    honour the disk type in the volume extra spec and adapter type in the image
    meta-data. The workflow to address these issues requires the above mentioned
    method. This change also sets the disk size of backing to at least 1MB which
    is the minimum required by VIM APIs.

    Closes-Bug: #1340315
    Partial-Bug: #1284284
    Partial-Bug: #1287185
    Partial-Bug: #1287176

    Change-Id: Icdb1f26d5c8eaa42408a14c57c7394d96dda078f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/106334
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=5eac161aa81a68e7cc4adb92798f6c3e849da81c
Submitter: Jenkins
Branch: master

commit 5eac161aa81a68e7cc4adb92798f6c3e849da81c
Author: Vipin Balachandran <email address hidden>
Date: Sun Jun 29 19:02:20 2014 +0530

    VMware: Volume from non-streamOptimized image

    Volume creation from non-streamOptimized images (preallocated/thin/sparse)
    has following problems:

    1) Sparse vmdk image is not converted to appropriate virtual disk type
       suitable for attaching to a nova instance.
    2) The adapter type in image meta-data is ignored while creating volumes.
    3) The vmware:vmdk_type extra_spec property is ignored.
    4) Virtual disk extent operation is called with a wrong parameter which
       might result in unwanted disk provisioning type conversion.

    This patch fixes the first 3 problems using the following workflow:

    a) Create a disk-less backing.
    b) Create a virtual disk (single flat extent) from the non-streamOptimized
       image
    c) Attach the virtual disk to the backing
    d) Clone the backing (if needed) to perform disk provisioning type conversion

    Closes-Bug: #1287176
    Closes-Bug: #1287185
    Closes-Bug: #1284284
    Change-Id: Ib7e9fae81d69d2fe490a4b603337f3d5cee1138c