Azure: cloud-init does not handle reformatting GPT partition ephemeral disks

[Stephen A. Zarkos]

=== Begin SRU Template ===
[Impact]
On Azure, cloud-init handles re-formatting the ephemeral disk.
The contents of the ephemeral disk for a system will be replaced with
a stock ephemeral disk in the following scenarios:
 a.) first boot
 b.) after a resize.
 c.) after a VM has been migrated from one host to another.

That ephemeral disk is either
 1. mbr partitioned with 1 ntfs partition
 2. gpt partitioned with 2 partitions, a msft reserved partition and a
    ntfs partition. This scenario is newer, and only occurs on
    larger instance types that have large ephemeral disks.

cloud-init previously did not handle '2' above.

[Test Case]
Generically this is re-creatable by:
 1.) launch an instance on Azure
 2.) resize it to a L32 or G5 size
 3.) check to see that the ephemeral disk (/dev/disk/cloud/azure_resource)
     has been formatted to ext4.

It is more easily recreated for testing and verification by:

1. launch instance on azure
2. re-partition the ephemeral disk to look like a "clean" disk above
3. remove old logs, reboot
   $ dir=logs-$(date +"%Y%m%d-%H%M%S");
   $ mkdir -p $dir; mv /var/log/cloud-init* $dir
4. ssh back in, expect that this the disk has an ext4 filesystem on it.
   And that it is mounted on /mnt.

   $ grep reformattable= /var/log/cloud-init.log
   2017-05-12 15:14:57,125 - DataSourceAzure.py[DEBUG]: reformattable=False: partition 1 (/dev/sdb1) on device /dev/disk/cloud/azure_resource was not ntfs formatted

   Or, if it was formatted, you'll see something like:
   2017-05-12 15:17:47,021 - DataSourceAzure.py[DEBUG]: reformattable=True: partition 2 (/dev/sdb2) on device /dev/disk/cloud/azure_resource was ntfs formatted and had no important files. Safe for reformatting.

   $ grep /mnt /proc/mounts
   /dev/sdb1 /mnt ext4 rw,relatime,data=ordered 0 0

[Regression Potential]
The change makes cloud-init accept another situation when it decides
to be reformat a disk. Reformatting of a disk could result in loss of
customer data if the decision to do so results in a false positive.

The fix came with some fairly extensive unit tests (TestCanDevBeReformatted)
on the 'can_dev_be_reformatted' method.

[Other Info]
Upstream commit at
  https://git.launchpad.net/cloud-init/commit/?id=31b6f1732

=== End SRU Template ===

Some Azure instances such as L32 or G5 have very large ephemeral disks which are partitioned via GPT vs. smaller ephemeral disks that have dos disklabels.

At first boot of an instance the ephemeral disk is prepared and formatted properly. But if the instance is deallocated and then reallocated (thus receiving a new ephemeral disk) then cloud-init does not handle reformatting GPT partition ephemeral disks properly. Therefore /mnt is never mounted again.

Test cases:
 1. Deploy an L32(s) VM on Azure
 2. Log in and ensure that the ephemeral disk is formatted and mounted to /mnt
 3. Via the portal you can "Redeploy" the VM to a new Azure Host (or alternatively stop and deallocate the VM for some time, and then restart/reallocate the VM).

Expected Results:
 - After reallocation we expect the ephemeral disk to be formatted and mounted to /mnt.

Actual Results:
 - After reallocation /mnt is not mounted and there are errors in the cloud-init log.

*This was tested on Ubuntu 16.04 - but may affect other releases.

Note: This bug a regression from previous cloud-init releases. GPT support for Azure ephemeral disk handling was added to cloud-init via this bug: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1422919.

Related bugs:
 * bug 1691489: fstab entries written by cloud-config may not be mounted

[Stephen A. Zarkos (stevez)]

Adding cloud-init logs. The logs show a number of boots (sorry), but you should just be able to look at the first two boots to see the issue.

[Stephen A. Zarkos (stevez)]

GPT formatted ephemeral disks have two partitions (unfortunately). The first is the Microsoft reserved partition (https://en.wikipedia.org/wiki/Microsoft_Reserved_Partition). This appears to be tripping up can_dev_be_reformatted().

Function address_ephemeral_resize() will first check if is_new_instance=True, and if so it will happily format the ephemeral disk. That's why this works at first boot.

For subsequent boots where is_new_instance=False, address_ephemeral_resize() will call can_dev_be_reformatted(). This function sees that the ephemeral disk has two partitions and returns false:

            if os.path.exists(devpath + suff + "2"):
                msg = ('device %s had more than 1 partition: %s, %s' %
                       devpath, cand, devpath + suff + "2")
                return False, msg

We also end up with the following TypeError in the logs:

 Traceback (most recent call last):
   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 647, in status_wrapper
     ret = functor(name, args)
   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 415, in main_init
     init.activate_datasource()
   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 369, in activate_datasource
     is_new_instance=self.is_new_instance())
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 251, in activate
     address_ephemeral_resize(is_new_instance=is_new_instance)
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 329, in address_ephemeral_resize
     result, msg = can_dev_be_reformatted(devpath)
   File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceAzure.py", line 279, in can_dev_be_reformatted
     devpath, cand, devpath + suff + "2")
 TypeError: not enough arguments for format string
.
.
Aside from the TypeError, even if we comment this part there is a later check to ensure that this partition is NTFS formatted (which it isn't). This will also return false and you will see the following error in the logs:

 "reformattable=False: partition 1 (/dev/disk/cloud/azure_resource-part1 -> /dev/sdb1) on device /dev/disk/cloud/azure_resource was not ntfs formatted"

I think the correct behavior should be for cloud-init to detect that this is a GPT formatted ephemeral disk, or check that azure_resource-part1 is a "Microsoft reserved partition". And if so skip azure_resource-part1 and focus on azure_resource-part2 instead to determine if it is safe to reformat.

[saim shrestha (saimsh)]

the issue has been reproduced for Ubuntu 14.04LTS images in azure L32S instance.

[Scott Moser (smoser)]

Hi,
Steven, Thanks for the good analysis.
I think we have a good understanding of the problem.

I'm working on getting a fix for the issue and also the TypeError that you pointed out.

[Scott Moser (smoser)]

I've linked an in-progress branch https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/323420
more work to do, but I think i have acceptable fixes there for the address_ephemeral_resize().

I think we still will need to modify somehow the builtin 'fs_setup' so that the disk_setup module also realizes that it can overwrite the ntfs partitions.

it currently shows:
    'fs_setup': [{'filesystem': 'ext4',
                  'device': 'ephemeral0.1',
                  'replace_fs': 'ntfs'}],

[Scott Moser (smoser)]

Stephen,
I believe the linked branch has a suitable fix for this issue now.

In response to my comment #6, I've realized that the 'replace_fs' in fs_setup configuration was always ignored, because if a partition is provided (in the "dotted" format there), then replace_fs is not used.

I've tested the general path on azure by re-formatting/partitioning the disk into something that looks like a newly partitioned disk and then rebooting.

[Stephen A. Zarkos (stevez)]

Hi Scott,

The linked branch seems to reliably format the ephemeral disk on large instances. However, after redeploying the VM I don't see the ephemeral disk mounted after boot. The cloud-init log does not indicate any error, and if I type 'mount -a' (or do a simple reboot) then the disk is mounted properly with an ext4 filesystem.

To test this I'm creating an L32s VM on Azure with Ubuntu 16.04, updating cloud-init manually from your git branch, and then I click the 'redeploy' option via the portal. I see that the ephemeral disk is partitioned/formatted correctly, but just not mounted. Am I missing some configuration? What is normally responsible for mounting the ephemeral disk once it is formatted?

Steve

[Scott Moser (smoser)]

Stephen,
I think this could be an issue that I saw as a possible issue a while ago, but never reproduced.
 https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/bug/before-fsck
has some details.

For what is normally responsible for mounting the ephemeral disk... its supposed to be just mounted by systemd's normal mounts. cc_mounts.py writes or cleans the entries in /etc/fstab, and then does a 'systemctl daemon-reload' with the intent of having systemd recognize realize the new entries and then handle everything.

I think we should probably treat that bug as separate than this one as it could happen anywhere independent of the disk format.

[Scott Moser (smoser)]

I've opened up bug 1691489 to address the mount/fsck race, and have been able to reproduce it and it looks like the fix i had in mind will work (more info there).

[Stephen A. Zarkos (stevez)]

Hi Scott,

As an additional test case I've tried to use the steps as described here to re-partition the ephemeral disk: https://wiki.ubuntu.com/AzureSwapPartitions.

Using your branch from this bug the sult is that it appears to fail to repartition but it does format/mount the disks. Here are my repro steps

 $ azure group create ubtest 'east us'

 $ azure vm create ubtest ubtest-1604 Linux --image-urn canonical:UbuntuServer:16.04-LTS:latest --admin-username azureuser --admin-password '<PASSWORD>' --nic-name ubtest-1604 --location "East US" --vnet-name "ubtest" --vnet-address-prefix "10.0.0.0/24" --vnet-subnet-name "ubtest" --vnet-subnet-address-prefix "10.0.0.0/24" --public-ip-name "ubtest" --public-ip-domain-name "ubtest" --custom-data ./customdata.txt -z 'Standard_L32s'

The customdata.txt file contains something like the following:

 #cloud-config
 disk_setup:
     ephemeral0:
         table_type: gpt
         layout: [66, [33, 82]]
         overwrite: True
 fs_setup:
     - device: ephemeral0.1
       filesystem: ext4
     - device: ephemeral0.2
       filesystem: swap
 mounts:
     - ["ephemeral0.1", "/mnt"]
     - ["ephemeral0.2", "none", "swap", "sw", "0", "0"]

.
.
Result with default 16.04 image: /mnt is not mounted and we see a failure in the cloud-init.log (attached).

Results with branch from this bug: /mnt and swap space are created, but the partition layout is not applied. So part1 is still type "Microsoft reserved" and only 120MB, and part2 is type "Microsoft basic data" and is a 5.5TB swap partition :)

Maybe there's something wrong with my cloud-config? Do I need to use a different layout format when table_type is 'gpt'?

[Scott Moser (smoser)]

Stephen,
I think the issue in your comment above is that cloud-init on the first boot wrote an entry into /etc/fstab for the swap partition. Then you upgraded rebooted. The /etc/fstab entry for swap was still present and systemd mounted swap. Then, when cloud-init went to repartition, the disk was busy (with swap) and could not get it done.

Could you attach *that* /var/log/cloud-init.log ?

[Scott Moser (smoser)]

Well, now I'm not so sure. I can reproduce this I think.
I'll open a separate bug tomorrow. The issue I think now is that 'check_partition_layout' is deciding that it does not need to repartition.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-144-g2825a917-0ubuntu1

---------------
cloud-init (0.7.9-144-g2825a917-0ubuntu1) artful; urgency=medium

  * New upstream snapshot.
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
      (LP: #1679817, #1685534, #1685532)
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - Fixing wrong file name regression. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - cloudstack: fix tests to avoid accessing /var/lib/NetworkManager
      [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present. [Chris Brinker]
      (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - Revert "tools/net-convert: fix argument order for render_network_state"
    - make deb: Add devscripts dependency for make deb. Cleanup packages/bddeb.
      [Chad Smith] (LP: #1685935)
    - tools/net-convert: fix argument order for render_network_state
      [Ryan Harper] (LP: #1685944)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network configuration defaults policy and formats.
      [Ryan Harper]
    - Fix name of "uri" key in docs for "cc_apt_configure" module
      [Felix Dreissig]
    - tests: Enable artful [Joshua Powers]

 -- Scott Moser <email address hidden> Tue, 23 May 2017 17:04:40 -0400

[Stephen A. Zarkos (stevez)]

Hi Scott,

Do you have a new bug number for the issue in comment #13?

Thanks,
Steve

[Scott Moser (smoser)]

Bug 1692087
I expect that this should be functional for you now. There are still some issues left that Paul has filed in bug 1692093.
I'll propose a merge for that tomorrow.

[Brian Murray (brian-murray)]

Hello Stephen, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Stephen, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Stephen, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Joshua Powers (powersj)]

This failed verification on an Azure L32s Zesty instance.

Steps to reproduce:
1. Booted an L32s instance.
2. Redeployed and verified that /mnt was not mounted.
3. Upgraded to the version of cloud-init to proposed, deleted /var/lib/cloud/, and rebooted.
4. Upon reboot /mnt was not mounted as expected and the cloud-init logs below show tracebacks.

cloud-init.log: https://paste.ubuntu.com/24920403/
cloud-init-output.log https://paste.ubuntu.com/24920406/

This did, however, work with Yakkety and Xenial.

[Joshua Powers (powersj)]

I am marking this verification-done-xenial and verification-done-yakkety as those tests passed.

Zesty is being marked verification-failed-zesty due to the above issue. I am working on reproducing and collecting additional details.

[Stephen A. Zarkos (stevez)]

I tested Xenial as well and verified that the cloud-init package in proposed resolves this bug.

[Joshua Powers (powersj)]

Zesty Steps to reproduce
1. Launch an L32s instance with 17.04 (Zesty)
2. Verify /mnt is mounted (mount or lsblk)
3. Update to proposed
4. From Azure console, redeploy the instance
5. Login and look if /mnt is mounted. If it is mounted, GOTO 4

I have seen it work properly before, so a second redeploy got the issue to appear.

cloud-init.log http://paste.ubuntu.com/24926325/
cloud-init-output.log http://paste.ubuntu.com/24926327/
journal http://paste.ubuntu.com/24926328/
syslog http://paste.ubuntu.com/24926329/

[Scott Moser (smoser)]

Josh,
The bug you see when it does not work properly is bug 1691489.
The issue as seen in your cloud-init.log is that when mkfs is run its check to see if the device is used fails. That is because the device is in use by the fsck process that is being run on filesystem that is already present there. systemd decides to run fsck on /dev/sdb1 because there is an entry (comment=cloudconfig) from the previous run.

[Joshua Powers (powersj)]

Because this resolves the original issues and the new issue is covered by bug 1691489 I am marking this as verification-done-zesty. Thanks Scott!

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~17.04.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~17.04.1) zesty-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network c...

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.10.1) yakkety-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network...

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network ...

[Scott Moser (smoser)]

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2872