[SRU] cloud-init needs backport of SmartOS w/ ephemeral disk support

Bug #1233315 reported by Ben Howard on 2013-09-30
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Medium
Ben Howard
Precise
Medium
Ben Howard
Quantal
Medium
Ben Howard
Raring
Medium
Ben Howard

Bug Description

[Impact]: Cloud-init now support SmartOS on Ubuntu 13.10, including the ability to format ephemeral devices. This SRU includes the SmartOS datasource, ephemeral disk format support, and enabling ephemeral format support in the Azure Datasource.

A backport of the SmartOS datasource is needed by the Canonical certified public cloud effort to support SmartOS based clouds.

[Test Case]: See Comment 4 for detailed test case information
SmartOS:
- Build image with SmartOS enabled as Cloud-init datasource
- Login via SSH
- Confirm:
  - /dev/vdb mounted to /mnt
  - /dev/vdb is of type ext3, label is 'ephemeral0'

Azure:
* Upgrade:
  - Boot current image
  - install new packages
  - confirm that previous /etc/waagent.conf has provisioning functions turned on
  - make sure that /var/lib/instance/sem/config-disk-setup is present
  - place file on ephemeral drive
  - reboot
  - confirm that file is on ephemeral drive
  - look at /var/lib/cloud-init.log indicates that disk_setup was skipped
* New Image:
  - Boot image built with new packages (i.e. from PPA/proposed)
  - confirm that /etc/waagent.conf has provisioning and ephemeral disk functions disabled
  - confirm that /dev/sdb1 mounted to /mnt
  - confirm that /dev/sdb1 is of type ext4, no label

[Regression]: This SRU adds a default cloud-init module of "disk_setup," which will format ephemeral devices for the Azure and SmartOS datasources. Both the Azure and SmartOS datasources define default devices that should be partitioned/formated.

There are a number of safe guards to prevent accidental data-deletion. The datasources are configured to _not_ overwrite any file system and to detect if the disk is in use.

As a further safe guard, the postinst sets a semaphore for cloud-init to prevent the disk_setup module from running when it detects that a system has been previously configured.

Ben Howard (darkmuggle) on 2013-09-30
affects: ubuntu → cloud-init (Ubuntu)
Scott Moser (smoser) on 2013-09-30
Changed in cloud-init (Ubuntu):
status: In Progress → Fix Released
Ben Howard (darkmuggle) on 2013-09-30
Changed in cloud-init (Ubuntu Precise):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Quantal):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Raring):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Precise):
assignee: nobody → Ben Howard (utlemming)
Changed in cloud-init (Ubuntu Quantal):
assignee: nobody → Ben Howard (utlemming)
Changed in cloud-init (Ubuntu Raring):
assignee: nobody → Ben Howard (utlemming)
Changed in cloud-init (Ubuntu Precise):
status: New → In Progress
Changed in cloud-init (Ubuntu Quantal):
status: New → In Progress
Changed in cloud-init (Ubuntu Raring):
status: New → In Progress
Ben Howard (darkmuggle) wrote :

Worked out the precise port, which needs to be reviewed. It works as expected and provisions the ephemeral storage. Branch is attached.

Ben Howard (darkmuggle) wrote :

Worked out raring port. It works as expected and provisions the ephemeral storage.

Ben Howard (darkmuggle) wrote :

The following is the test case:

Azure upgrade test:
1. Booted current version on Azure with default user of utlemming
2. Ran 'apt-get -y purge walinuxagent* cloud-int'
3. Installed PPA packages
4. Restored machine to pre-provisioning state:
   rm -rf /var/lib/cloud
   rm /etc/cloud/cloud.cfg.d/90-azure.cfg
   userdel -f utlemming
   :> /etc/fstab
   service walinuxagent stop
   umount /mnt/resource
   wipefs --all /dev/sdb1
   [P/Q]: mkfs.ntfs -f /dev/sdb1
5. Configure for cloud-init:
   [PRECISE ONLY]: create ubuntu user: useradd -m ubuntu -s /bin/bash
   dpkg-reconfigure cloud-init (selected SmartOS, removed others)
6. Run cloud-init
   cloud-init init
   [PRECISE ONLY]: cloud-init-cfg modules
   [Q/R]: cloud-init modules --mode=config
7. Confirmed that:
   - user 'utlemming' was created
   - user 'utlemming' has sudo
   - user 'utlemming' has ssh keys populated
   - /dev/sdb1 is mount to /mnt
   - /dev/sdb1 is of type ext4, no label

Azure new image test (SSH key test):
1. Build from PPA
2. Boot cloud image with user 'utlemming' defined via cloud
3. Log-in as user
4. Make sure that:
 - user 'utlemming' was created
 - user 'utlemming' has sudo
 - user 'utlemming' has ssh keys populated
 - /dev/sdb1 is mount to /mnt
 - /dev/sdb1 is of type ext4, no label
5. Check /var/log/cloud-init.log for any exceptions

Azure new image test (Password Authentication)
1. Build from PPA
2. Boot cloud image with user 'utlemming' defined via cloud
3. Log-in as user
4. Make sure that:
 - user 'utlemming' was created
 - user 'utlemming' has sudo
 - no SSH keys should be populated
 - /dev/sdb1 is mount to /mnt
 - /dev/sdb1 is of type ext4, no label
5. Check /var/log/cloud-init.log for any exceptions

SmartOS:
1. Booted SmartOS 12.04
2. Upgraded to Raring
3. Restored to pre-boot provisioning:
   - removed /data from /etc/fstab
   - umounted /data
   - wipefs --all /dev/vdb1
   - delpart /dev/vdb 1
4. Install cloud-init from PPA
5. Configure for cloud-init boot:
   -run: dpkg-reconfigure cloud-init (selected SmartOS, removed others)
6. Run 'cloud-init init'
   - confirmed that ubuntu users exists
   - confirmed that the ubuntu user has SSH keys
   - confirmed that datasource is selected
7. Run 'cloud-init modules'
    - confirmed that /mnt is in /etc/fstab and correct
 - confirmed that /dev/vdb mounted /mnt
 - confirmed that default filesystem on /dev/vdb is ext3
 - confirmed that the label is 'ephemeral0' on /dev/vdb
8. Rebooted and confirmed that ubuntu user can login
8. Confirm logs are as expected

summary: - cloud-init needs support for SmartOS
+ cloud-init needs backport of SmartOS w/ ephemeral disk support

Upgrade test for Azure:
1. Add PPA
2. Update
3. Run 'apt-get -y dist-upgrade'
4. Get a diff of the package waagent:
diff -u /etc/waagent.conf /etc/waagent.conf.dpkg-dist
--- /etc/waagent.conf 2013-10-14 11:26:20.128885001 +0000
+++ /etc/waagent.conf.dpkg-dist 2013-10-15 15:01:48.816663901 +0000
@@ -15,7 +15,7 @@
 Provisioning.SshHostKeyPairType=rsa # Supported values are "rsa", "dsa" and "ecdsa".
 Provisioning.MonitorHostName=n # Monitor host name changes and publish changes via DHCP requests.

-ResourceDisk.Format=y # Format if unformatted. If 'n', resource disk will not be mounted.
+ResourceDisk.Format=n # Format if unformatted. If 'n', resource disk will not be mounted.
 ResourceDisk.Filesystem=ext4
 ResourceDisk.MountPoint=/mnt
 ResourceDisk.EnableSwap=n # Create and use swapfile on resource disk.
5. The diff should show that ResourceDisk.Format=y is in /etc/waagent.conf
6. Put a file in /mnt
7. Reboot
8. The file should remain after boot (confirms that the disk is not formated

EC2 Test:
1. Add PPA
2. Update
3. Run 'apt-get -y dist-upgrade'
4. Place file in /mnt
5. Reboot
6. Confirm file in /mnt
7. Machine should reboot normally.

Ben Howard (darkmuggle) on 2013-10-17
summary: - cloud-init needs backport of SmartOS w/ ephemeral disk support
+ [SRU] cloud-init needs backport of SmartOS w/ ephemeral disk support
Ben Howard (darkmuggle) on 2013-10-17
description: updated

Hello Ben, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Precise):
status: In Progress → Fix Committed
tags: added: verification-needed
Stéphane Graber (stgraber) wrote :

Hello Ben, or anyone else affected,

Accepted cloud-init into raring-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.7.2-0ubuntu0.13.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Raring):
status: In Progress → Fix Committed
Ben Howard (darkmuggle) wrote :

Verified both upgrade and new image paths. Marking as verification-done.

Changed in cloud-init (Ubuntu Quantal):
status: In Progress → Won't Fix
tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.8

---------------
cloud-init (0.6.3-0ubuntu1.8) precise-proposed; urgency=low

  * debian/patches/lp-1233315-add-smartos-support.patch:
    Add SmartOS DataSource, including disk format support (LP: #1233315).
  * debian/patches/lp-1231490-azure-ephemeral-format.patch:
    Format Windows Azure ephemeral disks (LP: #1231490)
  * debian/patches/lp-1233315-1231490-ephmeralX.Y-support.patch:
    Add support for referencing ephemeral devices as 'ephemeralX.Y
    (LP: #1231490, LP: #1233315)
 -- Ben Howard <email address hidden> Wed, 16 Oct 2013 08:17:35 -0600

Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.2-0ubuntu0.13.04.2

---------------
cloud-init (0.7.2-0ubuntu0.13.04.2) raring-proposed; urgency=low

  [ Ben Howard ]
  * debian/patches/lp-1233315-1231490-smartos_azure_disk_format.patch:
    Backport from upstream 0.7.4:
    * SmartOS datasource to support booting on SmartOS Cloud targets
        (LP: #1233315)
    * Support for ephemeral drive formating for SmartOS
    * Enable default ephemeral formating for Windows Azure (LP: #1231490)
    * Support for shorthand on default block mappings, i.e. ephemeral0.1
      for both mounts and disk formatting.
    * Use partition on mount directives if available (LP: #1231490).
  * debian/cloud-init.postinst: ensure that disk-setup module is not run
    on upgrade.
 -- Scott Moser <email address hidden> Mon, 21 Oct 2013 18:00:50 -0700

Changed in cloud-init (Ubuntu Raring):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers