Azure ephemeral disk and custom mount points

[trialotto]

The creation of custom mount points with cloud-config, when creating a (linux) VM in Azure, is fairly simple, with some caveats.

However, the format

[ ephemeralX, <mount point> ]

is not working properly, even when explicitly declaring device aliases.

The format

[ sdb, <mount point> ]

or

[ sdbX, <mount point> ]

is working properly.

The above is not really a bug, but an inconvenience that can require some improvement.

In addition, note the following.

An "accidental" declaration of (something similar like)

[ swap, <mount point or "None"> ]

will result in the ephemeral disk not being mounted at all, even if a declaration of missing fields is applied.

The above is a bug, in the sense that a (mis)declaration in (custom) config settings should not result in an unmounted ephemeral disk.

The same results apply, when partitioning the ephemeral disk to contain a swap partition of type 82 and assigning the partition to a specific device (a change in fs_setup does not affect the results in any way), in order to have a possibility to declare

[ <swap device>, <mount point> ]

and therefore having a mounted disk/device with Linux swap partition.

The swap partition will not function as a swap space after boot and/or a swap file, when created at boot, on the swap partition will not be active.

In essence, the creation of a swap partition will lead to

a) only (!) the mount of the swap space (if the swap partition is declared first in the disk_setup, like for instance

disk_setup:
   ephmeral0:
       table_type: 'mbr'
       layout:
           - [33, 82]
           - 66
       overwrite: True
),

OR

b) the mount of the disk partition (if the swap partition is declared last), with the disk partition being smaller than the whole ephemeral disk (as result of the disk setup).

The above has only been executed for testing purposes, these scenarios do not have to occur in reality.

In conclusion, in creation of Azure (linux) VMs, two (potential) bugs exist:

- the failure to create custom mount points when referring to ephemeral0 (the proper name of the disk in Azure),
- the dangers of and/or issues with declarations regarding swap mounts.

Kind regards.....

[Stephen A. Zarkos (stevez)]

+1 :)

The [ sdb, <mount point> ] syntax is a bug as well. Device names are not persistent in Linux and could change, so it is not guaranteed that the ephemeral disk will be called /dev/sdb. Ideally this should be auto-detected in cloud-init at runtime (for example, see DeviceForIdePort() in the Azure Linux agent).

[trialotto (trialotto)]

@Stephen,

The non-persistence of device names is indeed a bug, even though device names can be declared by cloud-config settings.

With respect to the Azure Linux Agent, it should be noted that one has the option to use a standard declaration of device names (and partitions, if desired), instead of taking the "autodetection" approach.

The above option has the advantage that it allows for and/or can allow for

a - (persistent) fixation of device names,

b - (persistent) configuration of device names (and even partitions, if desired) in waagent.conf,

c - removal of multiple lines of code in the waagent script,

and it must be emphasized that the above implies that, in essence, the approach of using CustomData for injecting cloud-config settings becomes less relevant or even obsolete, if and only if the waagent script allows for using waagent.conf variables in the cloud-init sequence.

In short, there are some possibilities for the improvement of the waagent script and the proper functioning thereof.

Kind regards....

[Dan Watkins (oddbloke)]

Hi trialotto,

cloud-init has default settings for Azure instances that ensure that the first partition of the ephemeral disk is formatted as ext4. It does this by providing default values for disk_setup and fs_setup; you can see these at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceAzure.py#L54.

cloud-init, by default, mounts the first partition of the first ephemeral disk at /mnt. You can see the code that does this at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/config/cc_mounts.py#L236.

If you are changing the layout of the disk, you will also need to provide mount configuration (using the `mounts` configuration key) so that cloud-init knows what it should mount where. We could, potentially, look at making cloud-init a little smarter about automatically working out mounts, but this would require a fair bit of testing across different platforms.

Please let me know if this addresses your issues.

Thanks,

Dan

[Dan Watkins (oddbloke)]

Steve,

I have just filed https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1411582 regarding the inconstancy of device naming; could you confirm that I've captured the issue accurately?

Thanks,

Dan

[Dan Watkins (oddbloke)]

trialotto, I've just been doing some testing, and I've realised my above answer is incorrect; this is definitely a problem in how cloud-init's Azure data source works.

I'm looking in to it now.

[Dan Watkins (oddbloke)]

Actually, I've realised that I was passing in invalid cloud configuration data. If I have the following in azure-disk.cfg:

#cloud-config
disk_setup:
    ephemeral0:
        table_type: mbr
        layout: [66, 33]
        overwrite: True
fs_setup:
    - device: ephemeral0.1
      filesystem: ext4
    - device: ephemeral0.2
      filesystem: ext4
mounts:
    - ["ephemeral0.1", "/mnt", "auto", "defaults,nobootwait", "0", "0"]
    - ["ephemeral0.2", "/media", "auto", "defaults,nobootwait", "0", "0"]

and I run:

$ azure vm create ... --custom-data azure-disk.cfg

then I see the following:

$ ssh <email address hidden> df -h
Filesystem Size Used Avail Use% Mounted on
...
/dev/sdb1 393G 71M 373G 1% /mnt
/dev/sdb2 203G 60M 192G 1% /media

I think there _is_ a problem with specifying swap partitions (they don't seem to get `mkswap` run on them before a mount is attempted). That will be handled in bug 1374166, so I'm going to mark this as Invalid and we can track the swap issues over there.

[trialotto (trialotto)]

Daniel,

I was aware of DataSourceAzure.py and cc_mounts.py and the fact that built-in config for Azure requires declaration of "sdb" (and not "ephemeral0"). The before mentioned fact is only an inconvenience for users that would like to declare custom mount points and make the mistake to use an in incorrect reference to the ephemeral disk (i.e. not declaring "sdb"). The misdeclaration will only result in the ephemeral disk not being mounted, that should not be a real issue.

However, the above reveals a (minor) bug in cc_mounts.py.

The cfg mnt array in the cc_mounts.py script is sanitized, in such a fashion that default mounts are only added if other entries do not have the same device name (see lines 285 - 307).

A declaration of [ sdb, <mount point> ] or [ sdbX, <mount point> ] (in case of partitions) in custom cloud config settings will not result in an append of the ephemeral0 default mount (since the device name remains the same) and that is good.

After all, when declaring [ sdb, <mount point> ] or [ sdbX, <mount point> ] (in case of partitions), cc_mounts.py will append device "/dev/sdb" or "/dev/sdbx" (in case of partitions) to the cfg mnt arry, instead of the default mount declarations for ephemeral0.

This is really cumbersome, when using partitions and certainly when using swap (dedicated) partitions.

The bug is (in essence) that the sanitizing in cc_mounts.py does not prevent that the standard alias "/dev/sdb", as specificied in DataSourceAzure.py, or any other declared aliases will be appended to the cfg mnt array, which should not occur, if these aliases are pointing to one and the same ephemeral disk.

The (minor) bug can be easily resolved by sanitizing for aliases in cc_mounts.py.

I was (also) aware that cc_mounts.py contains the declaration ["swap", "none", "swap", "sw", "0", "0"] in the default set of mounts and that DataSourceAzure.py does not have to contain any reference to swap or mount points thereof.

However, the "accidental" declaration of [ swap, <mount point or "None"> ] reveals a bug, that occurs even when a declaration of the form [ sdb, <mount point> ] is provided in cloud-config settings.

The bug is, in my humble opinion, (solely) related to the following lines of code in cc_mounts.py:

for line in actlist:
        # write 'comment' in the fs_mntops, entry, claiming this
        line[3] = "%s,%s" % (line[3], comment)
        if line[2] == "swap":
            needswap = True
        if line[1].startswith("/"):
            dirs.append(line[1])
        cc_lines.append('\t'.join(line))

with these lines of code stating that any device called swap (or /dev/swap, since that is the same in cloud-init)

a) will mount, as a result of

        if line[1].startswith("/"):
            dirs.append(line[1])

if only if

- the declaration is of form [ swap, <mount point> ] AND
- the device called swap actually exists, for instance as a result of the existence of swap partitions AND
- the swap partition is declared first in the disk_setup cloud-config settings,

b) will not mount, as a result of (taking into account defvals = [None, None, "auto", "defaults,nobootwait", "0", "2"])

       if line[2] ==...

[trialotto (trialotto)]

@Daniel,

I just read your latest (two) posts, I was not able to read them while commenting, my apologies.

The "makeswap" bug (when using partitions) is very different from the "swap" bug and the "alias" bug.

In short:

- the "swap" bug can be resolved by sanitizing declarations of form [ swap, x, x, ... ]
- the "alias" bug can be resolved by adding some code to check for double entries in the cfg mgt array, due to aliases

Kind regards.....