need config-drive-v2 support

[Scott Moser]

== Begin SRU Information ==
[Impact]
'config-drive' is a mechanism for passing data from the hypervisor (or
cloud platform) to the guest (instance). config-drive-v2 was implemented
in Openstack Folsom release, and delivered as functional in 12.10 Ubuntu.

In order to support Ubuntu 12.04 cloud images running as a guest on
Openstack clouds that utilize config-drive, we need to backport this
function to 12.04.

This feature backport can be considered "hardware enablement" as it
enables Ubuntu images to work on new cloud platforms.

[Test Case]
Attached to this bug is an ISO that provides config-drive-v2 data.
Currently, if you attach the ISO to a cloud-image instance,
it will be ignored.

The new version of cloud-init enables data to be read from the disk.

The provided ISO file simply sets a password for the 'ubuntu' user to
'ubuntu'. So, verification that the test worked is as easy as logging
in with 'ubuntu' and 'passw0rd', either via ssh or via the console.
Additionally, when the config-drive is not found, cloud-init will fall
back to the EC2 data source, and block boot for several minutes as it
waits. When the config-drive is found, boot will occur quickly.

To perform this test, download a precise cloud-image from
http://cloud-images.ubuntu.com patch it, and boot it with kvm.

$ imgurl="http://cloud-images.ubuntu.com/releases/precise/release-20130124/ubuntu-12.04-server-cloudimg-amd64-disk1.img"
$ deburl="https://launchpad.net/~smoser/+archive/cloud-init-test/+files/cloud-init_0.6.3-0ubuntu1.5%7Eppa1_all.deb"
$ isourl="https://bugs.launchpad.net/cloud-init/+bug/1037567/+attachment/3509468/+files/disk.config.gz"

$ wget $imgurl -O precise-amd64.img.dist
$ wget $deburl -O cloud-init.deb
$ wget $isourl -O cfgdisk.img.dist;

$ qemu-img convert -O qcow2 precise-amd64.img.dist disk1.img.dist
$ qemu-img create -f qcow2 -b disk1.img.dist patched.img.dist

$ zcat --force cfgdisk.img.dist > cfgdisk.img
$ chmod 600 cfgdisk.img disk1.img.dist

# patch the patched.img.dist with new cloud-init
$ bzr branch lp:~smoser/+junk/backdoor-image ./bi
$ sudo ./bi/mount-callback-umount patched.img.dist -- \
    sh -ec 'mp=$1; cp cloud-init.deb $mp/tmp &&
            LANG=C chroot $mp dpkg -i /tmp/cloud-init.deb ;
            rm $mp/tmp/cloud-init.deb' --

$ qemu-img create -f qcow2 -b disk1.img.dist unpatched.img
$ qemu-img create -f qcow2 -b patched.img.dist patched.img

# boot patched and unpatched images
$ kvm -m 512 -drive file=unpatched.img,if=virtio -drive file=cfgdisk.img,if=virtio
$ kvm -m 512 -drive file=patched.img,if=virtio -drive file=cfgdisk.img,if=virtio

The unpatched version will take quite a long time to boot, and you'll messages
on the serial console like:see:
 2013-01-31 18:53:18,185 - DataSourceEc2.py[CRITICAL]: giving up on md after 120

[Regression Potential]
The potential for regression is low. The most likely possibility for
error would be in incorrectly identifying a cd-rom and its content as a
config-drive.
== End SRU Information ==

## Howto launch instance with config-drive ###
nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey
nova boot --key_name=mykey --image=$IMAGE \
   --config-drive=1 --flavor=m1.tiny \
   --user_data=./user-data.txt cfg-drive-test
We need to update the config-drive datasource to support config-drive-v2 [1]. There is a document showing what the config-drive looks like at [2].

[1] https://blueprints.launchpad.net/nova/+spec/config-drive-v2

[Scott Moser (smoser)]

I've committed changes for this in a precise branch at lp:~smoser/ubuntu/precise/cloud-init/sru . I have a ppa build of that at https://launchpad.net/~smoser/+archive/cloud-init-test/ . Any testing on that would be appreciated.

The plan is to move SRU this as soon as the current SRU moves to -updates.

[Scott Moser (smoser)]

Alex has requested that any work done here support reading data from a cdrom as well as from a disk drive.

[Clint Byrum (clint-fewbar)]

Hello Scott, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Scott Moser (smoser)]

Verified by booting most recent released cloud-image for 12.04 under raring kvm.

$ imgurl="http://cloud-images.ubuntu.com/releases/precise/release-20130204/ubuntu-12.04-server-cloudimg-amd64-disk1.img"
$ deburl="https://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.5/+build/4312778/+files/cloud-init_0.6.3-0ubuntu1.5_all.deb"
$ isourl="https://bugs.launchpad.net/cloud-init/+bug/1037567/+attachment/3509468/+files/disk.config.gz"

$ wget $imgurl -O precise-amd64.img.dist
$ wget $deburl -O cloud-init.deb
$ wget $isourl -O cfgdisk.img.dist;

$ qemu-img convert -O qcow2 precise-amd64.img.dist disk1.img.dist
$ qemu-img create -f qcow2 -b disk1.img.dist patched.img.dist

$ zcat --force cfgdisk.img.dist > cfgdisk.img
$ chmod 600 cfgdisk.img disk1.img.dist

## patch the patched.img.dist with new cloud-init
$ bzr branch lp:~smoser/+junk/backdoor-image ./bi
$ sudo ./bi/mount-callback-umount patched.img.dist -- \
    sh -ec 'mp=$MOUNTPOINT; cp cloud-init.deb $mp/tmp &&
            LANG=C chroot $mp dpkg -i /tmp/cloud-init.deb ;
            rm $mp/tmp/cloud-init.deb' --

$ qemu-img create -f qcow2 -b disk1.img.dist unpatched.img
$ qemu-img create -f qcow2 -b patched.img.dist patched.img

# boot patched and unpatched images
$ kvm -m 512 -drive file=unpatched.img,if=virtio -drive file=cfgdisk.img,if=virtio
$ kvm -m 512 -drive file=patched.img,if=virtio -drive file=cfgdisk.img,if=virtio

[Colin Watson (cjwatson)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.5

---------------
cloud-init (0.6.3-0ubuntu1.5) precise-proposed; urgency=low

  * debian/update-grub-legacy-ec2: consider kernels bootable on ec2
    that are named -generic, in addition to -virtual. This fixes a problem
    where the kernels installed by linux-lts-quantal were not added to
    /boot/grub/menu.lst (LP: #1005551)
  * debian/patches/lp-1077020-fix-ca-certificates-blanklines.patch: fix
    adding of empty lines in ca-certificates file (LP: #1077020)
  * debian/patches/lp-1031065-nonet-not-start-networking.patch: do not 'start
    networking' in cloud-init-nonet upstart job. Doing so can cause networking
    to be started earlier than it should be. Instead, add a
    cloud-init-container job that runs only in a container and emits
    net-device-added (LP: #1031065).
  * debian/patches/lp-1037567-add-config-drive-v2-support.conf:
    backport support for config-drive-v2 which is part of Openstack Nova in
    Folsom and later. (LP: #1037567) (LP: #1100545)
 -- Scott Moser <email address hidden> Wed, 16 Jan 2013 19:37:57 -0500

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2281