disk_setup could deal with non-deterministic device naming on EC2 nitro instances

Bug #1828611 reported by Nils Meyer
This bug affects 2 people
Affects Status Importance Assigned to Milestone

Bug Description

Cloud Provider: Amazon Web Services

As is documented in [1], instances on the nitro type hypervisor don't attach the NVME disks in deterministic order, yielding a different order of disks, example [2]. This makes it somewhat difficult to format and partition volumes since you don't know the volume ids beforehand when creating an instance (in an Autoscaling group for example).

My current thinking is that maybe a sort of special device name (much like swap / ephemeralX) coul be used to locate a device, for example ebs:root for the root drive - which is easy to detect, ebs:size=12G[0] for the first volume found with 12GiB size. With an appropriate instance profile and boto3 more elaborate selectors are conceivable (for example based on tags).

Further complicating things is that the metadata endpoint doesn't expose the correct device names, opting instead for fantasy names (sda1 for the root volume, sdX for other volumes).

My Workaround for a 2 volume instance: Try and format both devices, then mount the disk by label:
  - label: mylabel
    device: /dev/nvme0n1
    filesystem: xfs
    partition: none
    overwrite: false
  - label: mylabel
    device: /dev/nvme1n1
    filesystem: xfs
    partition: none
    overwrite: false

- ["/dev/disk/by-label/mylabel", "/mnt/label", "xfs", "defaults"]

[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html
[2] https://gist.github.com/nilsmeyer/eddcfa4b7fc5b04ebc0be9eaa3c7b7dd

Revision history for this message
Ryan Harper (raharper) wrote :

Thanks for filing the issue and providing your workaround.

Changed in cloud-init:
importance: Undecided → Low
status: New → Triaged
Revision history for this message
Jan Brouwers (janbrouwers) wrote :

running into the same issue.
The workaround does not work for me as I need 2 drives formatted with different filesystems.

Worse, there doesn't seem to be a way to identify the drives from the ec2 metadata.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.