no ideal way to boot an ubuntu cloud-image on openstack

[Scott Moser]

The current path for "upload an Ubuntu image" for nova goes something like this:
 $ cloud-publish-tarball oneiric-server-cloudimg-i386.tar.gz oneiric i386
alternatively
 $ cloud-publish-image \
     --kernel-file oneiric-server-cloudimg-vmlinuz-virtual \
      i386 oneiric-server-cloudimg-i386.img oneiric \

The problem with either of these is that they result in libvirt/nova invoking kvm to load the kernel as '--kernel'. When that is done, kvm will then re-load the kernel on instance reboot. The result is that inside an instance 'apt-get dist-upgrade' that pulls in a kernel package has no effect after reboot.

We have addressed this shortcoming on Eucalyptus and on EC2.
 a. On Eucalyptus, under bug 611144, I added support for loading a "loader" rather than a kernel. We exposed this functionality in cloud-publish-tarball via '--loader' flag.
 b. On eucalyptus we've also made available a "loader" kernel and ramdisk, which do nothing more than boot, mount the root filesystem and the kexec into the real kernel. (https://code.launchpad.net/~smoser/+junk/kexec-loader)
 c. On EC2 we're using 'pv-grub' as the loader, and grub-legacy-ec2 to load the kernel.

Both of those mean that after 'apt-get dist-upgrade && reboot' the user will boot into a new kernel.

On nova, our options at the moment would appear to be:
 1. re-use the kexec-loader as described in 'b' above.
    This would work, but the kexec-loader is less "complete" a loader compared to grub. It has only very limited support for parsing a grub.cfg file to find the kernel to load.

 2. create new full-disk images for nova (with a partition table and grub-pc installed in the MBR)
    This means carrying yet another deliverable in cloud-images. The most user friendly format would be compressed raw image. These images would then not be suitable (without modification to tools) as a replacement for the existing partition images, as they then could not be directly uploaded to EC2 or Eucalyptus.

 3. suggest use of vmdk files that we're currently producing. (currently broken at bug 826798).
   The issue with vmdk format is it is not linux friendly. You *can* convert the sparse compressed images from vmdk to "raw" using virtual box tools, but you cannot do so with qemu-img. In general, they're less immediately useful format.

[Scott Moser (smoser)]

Some updates.
 * The latest builds [3] (20110829) have a .qcow2 file which is a compressed qcow disk image.
 * I posted an email to 2 ubuntu-cloud [1] and ec2ubuntu [2] asking if anyone is using the .vmdk files. I'm thinking of replacing them with compressed qcow images.

The compressed qcow images do work with:
 uec-publish-image amd64 oneiric-server-cloudimg-amd64-disk1.qcow2 $BUCKET -v --type image

However, they're still less than ideal. It seems that nova is not resizing the disk image up as it does with a partition image (bug 836759).

--
[1] https://lists.ubuntu.com/archives/ubuntu-cloud/2011-August/thread.html#655
[2] https://groups.google.com/group/ec2ubuntu/browse_frm/thread/69d7a94e32642b1b
[3] https://cloud-images.ubuntu.com/server/oneiric/current/

[Scott Moser (smoser)]

With the Nova diablo release (packaged in Ubuntu Oneiric beta-2) the full disk images available on http://cloud-images.ubuntu.com are in an ideal form for use in nova.
These are full disk compressed qcow2 images. After bug 837100 was fixed, the images are uncompressed before usage.

So, the correct way to use Ubuntu oneiric images is to do:
  $ cloud-publish-image i386 oneiric-server-cloud-i386.img bucket/

If you want to run older releases of Ubuntu and support kernel upgrades the best path for that is to upload a loader kernel as is available at [1]

--
[1] http://people.canonical.com/~smoser/lucid-loaders/

[Ryan Lane (rlane)]

Note: to use the kexec-loader with glance commands, do the following:

/usr/bin/glance-upload --disk-format=ari --container-format=ari --type=ramdisk loader-ramdisks/<loader> loader-ramdisks/<loader>

(note the number)

/usr/bin/glance-upload --disk-format=aki --container-format=aki --type=kernel ubuntu-kernels/<kernel> ubuntu-kernels/<kernel>

(note the number)

/usr/bin/glance-upload --disk-format=ami --container-format=ami --type=machine --kernel=<kernel-number> --ramdisk=<ramdisk-number> <img> <img>

(note the number)

/usr/bin/glance update <kernel-number> type=kernel version=<version> distro=ubuntu uploader=<your-email> arch=<arch>

/usr/bin/glance update <img-number> type=machine version=<version> distro=ubuntu uploader=<your-email> arch=<arch> kernel_id=<kernel-number> kernel_name=ubuntu-kernels/<kernel> ramdisk_id=<ramdisk-number> ramdisk_name=loader-ramdisks/<ramdisk>

/usr/bin/glance update <ramdisk-number> type=ramdisk distro=ubuntu uploader=<your-email> arch=<arch>