Azure: cloud-init does not handle reformatting GPT partition ephemeral disks
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cloud-init |
Fix Released
|
Medium
|
Scott Moser | ||
cloud-init (Ubuntu) |
Fix Released
|
High
|
Scott Moser | ||
Xenial |
Fix Released
|
Medium
|
Unassigned | ||
Yakkety |
Fix Released
|
Medium
|
Unassigned | ||
Zesty |
Fix Released
|
Medium
|
Unassigned | ||
Artful |
Fix Released
|
High
|
Scott Moser |
Bug Description
=== Begin SRU Template ===
[Impact]
On Azure, cloud-init handles re-formatting the ephemeral disk.
The contents of the ephemeral disk for a system will be replaced with
a stock ephemeral disk in the following scenarios:
a.) first boot
b.) after a resize.
c.) after a VM has been migrated from one host to another.
That ephemeral disk is either
1. mbr partitioned with 1 ntfs partition
2. gpt partitioned with 2 partitions, a msft reserved partition and a
ntfs partition. This scenario is newer, and only occurs on
larger instance types that have large ephemeral disks.
cloud-init previously did not handle '2' above.
[Test Case]
Generically this is re-creatable by:
1.) launch an instance on Azure
2.) resize it to a L32 or G5 size
3.) check to see that the ephemeral disk (/dev/disk/
has been formatted to ext4.
It is more easily recreated for testing and verification by:
1. launch instance on azure
2. re-partition the ephemeral disk to look like a "clean" disk above
3. remove old logs, reboot
$ dir=logs-$(date +"%Y%m%d-%H%M%S");
$ mkdir -p $dir; mv /var/log/
4. ssh back in, expect that this the disk has an ext4 filesystem on it.
And that it is mounted on /mnt.
$ grep reformattable= /var/log/
2017-05-12 15:14:57,125 - DataSourceAzure
Or, if it was formatted, you'll see something like:
2017-05-12 15:17:47,021 - DataSourceAzure
$ grep /mnt /proc/mounts
/dev/sdb1 /mnt ext4 rw,relatime,
[Regression Potential]
The change makes cloud-init accept another situation when it decides
to be reformat a disk. Reformatting of a disk could result in loss of
customer data if the decision to do so results in a false positive.
The fix came with some fairly extensive unit tests (TestCanDevBeRe
on the 'can_dev_
[Other Info]
Upstream commit at
https:/
=== End SRU Template ===
Some Azure instances such as L32 or G5 have very large ephemeral disks which are partitioned via GPT vs. smaller ephemeral disks that have dos disklabels.
At first boot of an instance the ephemeral disk is prepared and formatted properly. But if the instance is deallocated and then reallocated (thus receiving a new ephemeral disk) then cloud-init does not handle reformatting GPT partition ephemeral disks properly. Therefore /mnt is never mounted again.
Test cases:
1. Deploy an L32(s) VM on Azure
2. Log in and ensure that the ephemeral disk is formatted and mounted to /mnt
3. Via the portal you can "Redeploy" the VM to a new Azure Host (or alternatively stop and deallocate the VM for some time, and then restart/reallocate the VM).
Expected Results:
- After reallocation we expect the ephemeral disk to be formatted and mounted to /mnt.
Actual Results:
- After reallocation /mnt is not mounted and there are errors in the cloud-init log.
*This was tested on Ubuntu 16.04 - but may affect other releases.
Note: This bug a regression from previous cloud-init releases. GPT support for Azure ephemeral disk handling was added to cloud-init via this bug: https:/
Related bugs:
* bug 1691489: fstab entries written by cloud-config may not be mounted
Related branches
- Scott Moser: Approve
- Server Team CI bot: Approve (continuous-integration)
- Ryan Harper: Needs Fixing
-
Diff: 588 lines (+307/-77)4 files modifiedcloudinit/config/cc_disk_setup.py (+14/-5)
cloudinit/sources/DataSourceAzure.py (+49/-35)
tests/unittests/test_datasource/test_azure.py (+228/-37)
tests/unittests/test_handler/test_handler_disk_setup.py (+16/-0)
Changed in cloud-init (Ubuntu): | |
assignee: | nobody → Scott Moser (smoser) |
Changed in cloud-init (Ubuntu): | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in cloud-init: | |
status: | New → Confirmed |
importance: | Undecided → Medium |
Changed in cloud-init (Ubuntu): | |
importance: | Medium → High |
status: | Confirmed → In Progress |
description: | updated |
Changed in cloud-init: | |
status: | In Progress → Fix Committed |
assignee: | nobody → Scott Moser (smoser) |
Changed in cloud-init (Ubuntu Xenial): | |
status: | New → Confirmed |
Changed in cloud-init (Ubuntu Yakkety): | |
status: | New → Confirmed |
Changed in cloud-init (Ubuntu Zesty): | |
status: | New → Confirmed |
Changed in cloud-init (Ubuntu Xenial): | |
importance: | Undecided → Medium |
Changed in cloud-init (Ubuntu Yakkety): | |
importance: | Undecided → Medium |
Changed in cloud-init (Ubuntu Zesty): | |
importance: | Undecided → Medium |
description: | updated |
Adding cloud-init logs. The logs show a number of boots (sorry), but you should just be able to look at the first two boots to see the issue.