Lack of EC2 image attributes for volume backed snapshot.

[Feodor Tersin]

For EBS images AWS returns device names, volume sizes, delete on termination flags in block device mapping structure.

$ euca-describe-images ami-d13845e1
IMAGE ami-d13845e1 amazon/amzn-ami-hvm-2014.03.2.x86_64-ebs amazon available public x86_64 machine ebs hvm xen
BLOCKDEVICEMAPPING /dev/xvda snap-d15cde24 8 true

The same in xml form:
            <blockDeviceMapping>
                <item>
                    <deviceName>/dev/xvda</deviceName>
                    <ebs>
                        <snapshotId>snap-d15cde24</snapshotId>
                        <volumeSize>8</volumeSize>
                        <deleteOnTermination>true</deleteOnTermination>
                        <volumeType>standard</volumeType>
                    </ebs>
                </item>
            </blockDeviceMapping>

But Nova didn't do it now:

$ euca-describe-images ami-0000000a
IMAGE ami-0000000a None (sn-in) ef3ddd7aa4b24cda974200baef02730b available private machine aki-00000002 ari-00000003 instance-store
BLOCKDEVICEMAPPING snap-00000005

The same in xml form:
      <blockDeviceMapping>
        <item>
          <ebs>
            <snapshotId>snap-00000005</snapshotId>
          </ebs>
        </item>
      </blockDeviceMapping>

NB. In Grizzly device names and delete on termination flags was returned. It was changed by https://github.com/openstack/nova/commit/33e3d4c6b9e0b11500fe47d861110be1c1981572
Now these attributes are not stored in instance snapshots, so there is no way to output them.

Device name is most critical attribute. Because there is another one compatibility issue (see https://bugs.launchpad.net/nova/+bug/1370250): Nova isn't able to adjust attributes of volume being created at instance launch stage. For example in AWS we can change volume size and delete on termination flag of a device by set new values in parameters of run instance command. To identify the device in image block device mapping we use device name. For example:
euca-run-instances ... -b /dev/vda=:100
runs an instance with vda device increased to 100 GB.
Thus if we haven't device names in images, we haven't a chance to fix this compatibility problem.

[Sean Dague (sdague)]

Which version of OpenStack is this on, just to callibrate.

[Feodor Tersin (ftersin)]

Device name and delete on termination flag are not reported in current version (Juno) and Icehouse. They are reported in Havana and Grizzly.
Volume size is not reported in all versions known for me (since Grizzly).

[Feodor Tersin (ftersin)]

There is another one issue on image attributes
https://bugs.launchpad.net/nova/+bug/1384386

[Nikola Đipanov (ndipanov)]

So after looking at the code - the delete-on-termination flag and volume_size were likely dropped by mistake, and putting it back should be easy, but there could be images that do not have this information so ec2-api code will likely need to be patched to default it somehow.

Device name is a little bit trickier, since we also need to make sure that booting works both with images that do and do not have block device mapping with device names. (it also ties in to https://bugs.launchpad.net/nova/+bug/1370250)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/186910

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/186911

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/186910
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=aa2f5dcc5f2a4666601905af6b6318da05a8c34e
Submitter: Jenkins
Branch: master

commit aa2f5dcc5f2a4666601905af6b6318da05a8c34e
Author: Nikola Dipanov <email address hidden>
Date: Fri May 29 20:59:10 2015 +0100

    bdm: Make sure that delete_on_termination is a boolean

    Make sure that 'delete_on_termination' is always a boolean, even if it
    was not passed in and thus got defaulted to None up until now.

    This patch is really more of a pedantic fix, as we use objects for BDMs
    everywhere outside of the API service, and object fields are of the
    correct type.

    Related-bug: #1370177

    Change-Id: I2724bcbe159490f3bdd85f833412f3b20c4e1e23

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/186911
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a7de013891da5afd43c2aa2dd6dad61048799230
Submitter: Jenkins
Branch: master

commit a7de013891da5afd43c2aa2dd6dad61048799230
Author: Nikola Dipanov <email address hidden>
Date: Fri May 29 21:12:16 2015 +0100

    snapshot: Copy some missing attrs to the snapshot bdms

    The following commit: 33e3d4c6b9e0b11500fe47d861110be1c1981572 wrongly
    drops some of the block device mapping information that is needed to
    fully describe the block device mapping. In addition - EC2 API needs
    this information to be able to format the output.

    Change-Id: I18a0aab1dd966f28973e1f3705f263cb46a42639
    Partial-bug: #1370177

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/188914

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/188915

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/190323

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/198313
Committed: https://git.openstack.org/cgit/stackforge/ec2-api/commit/?id=aeb2396905d52024b1b99545c8eefc52d6a62c83
Submitter: Jenkins
Branch: master

commit aeb2396905d52024b1b99545c8eefc52d6a62c83
Author: Feodor Tersin <email address hidden>
Date: Fri Jul 3 14:09:51 2015 +0300

    Enable check of volume size in ebs image test

    Since LP #1370177 is partially fixed by
    https://review.openstack.org/#/c/188914/

    Related-Bug: #1370177
    Depends-On: I9ffbb6cf40c1012f774a382cf1dcceba64637e6d
    Change-Id: I282f41e5c61b85e10cb86b63c51e67d0da4fce56

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/199412
Committed: https://git.openstack.org/cgit/stackforge/ec2-api/commit/?id=5533ca721d772aa51d622ec46254050addeb1b0a
Submitter: Jenkins
Branch: master

commit 5533ca721d772aa51d622ec46254050addeb1b0a
Author: Feodor Tersin <email address hidden>
Date: Wed Jul 8 09:32:53 2015 +0300

    Add functional test to check not root bdm in ebs image

    Related-Bug: #1370177
    Depends-On: Ib1ba130042aabbbe7bb8d60fc212c66e446c1d73
    Change-Id: Ica7a09c4beb52bb7341c6640e81da80133e56cf1