Allow cloud-config to modify partitioning and mount point of temporary resource disk in Azure

[Stephen A. Zarkos]

Users would like to use a local cloud-config to modify how the resource/ephemeral disk is partitioned and where it is mounted on Azure VMs. Currently it seems this is only customizable via a configuration sent via CustomData (user data).

If I understand correctly, the Azure data source overrides the local cloud config for how this disk is set up. Users would like to be able to use a local cloud-config to override what's defined in the data source.

[trialotto (trialotto)]

The mount location can be reconfigured in the waagent.conf file, after provisioning of the VM.

A shut-down and start-up sequence is required, in order to let the changed settings become effective.

It is not desirable and/or advisable to change partitioning and/or mount locations on creation time.

It is also not desirable and/or advisable to change partitioning of the resource (ephemeral) disk, given the fact that this disk is not really intended for persistent data storage and the fact that one minor change in settings (changing ResourceDisk.Format=n to ResourceDisk.Format=y) will result in a loss of all data and partitioning settings (!).

When requiring a (persistent) data disk, please attach an additional data disk (with all freedom available, for custom configuration of partitioning and/or mount points).

Kind regards....

[Stephen A. Zarkos (stevez)]

No, the mount location cannot be modified on Ubuntu via the waagent.conf, that is the issue. I would like a way to modify the mount point post-provisioning.

The only way to change the mount point on Azure is to use CustomData (like "user data" on other clouds) to send a cloud-config to modify the mount point at provision time. This is on par with with other cloud providers - the ephemeral disk can be repartitioned and/or mounted at a different location at provisioning time.

[trialotto (trialotto)]

@Stephen,

The mount location CAN be changed by setting

- ResourceDisk.Format=y (instead of "n") AND
- ResourceDisk.MountPoint=/opt (or any other directory, instead of /mnt)

in waagent.conf.

The change becomes effective after a shutdown and start-up sequence (and I assume the same applies to a normal restart).

The mount location cannot be changed, if and only if ResourceDisk.Format=n.

Also note that the ResourceDisk.MountPoint variable in waagent.conf can be blank, in which case the standard mount point will be used by cloud-init, being "/mnt".

All the above already leaves the question why ResourceDisk.MountPoint should be present in waagent.conf, given that

- most of the work is done by cloud-init and cloud-config settings,
- the ResourceDisk.MountPoint is only working in combination with ResourceDisk.Format=y,
- the specification of mount points should not be limited to only the resource (ephemeral) disk,

and so on.

In essence, the Azure Datasource, used by cloud-init, should be able to push some flexible configuration of variables to cloud-init.

As a result, a desire to change many variables will require that

1 - a (huge) number of variables is added to waagent.conf, with corresponding code in waagent script, OR
2 - the Azure Datasource should be WRITEABLE, allowing the change of cloud-config settings without specifying CustomData at provisioning time and (therefore) the changed cloud-config settings becoming effective at shut-down and start-up sequences,

and, to be honest, the proper and efficient way of doing things should be method 2.

Kind regards...

[Stephen A. Zarkos (stevez)]

That's what I meant, on Ubuntu images the ResourceDisk.Format should be "n" because cloud-init handles this. We do not want the Linux agent and cloud-init fighting over formatting or reformatting, and then mounting the resource disk.

You're right, perhaps these parameters should not exist in the cloud images. That would be a simple patch to the walinuxagent package unless cloud-init wants to start using those parameters at some point.

Thanks.

[trialotto (trialotto)]

@Stephen,

I think you know why I stated that the ResourceDisk.Format variable is a remarkable explanation for specific behaviour of waagent.

It must also be clear why I (also) stated that some parts of the waagent script are rather obsolete or unnecessary.

A huge part of the waagent script can be discarded when allowing cloud-init to do the work, including those parts of the waagent script that are aiming on specific distro´s, since cloud-init also takes care of distro-specific code.

However, allowing cloud-init do to all the work, will only become a huge advantage if

1 - the corresponding (Azure) Datasource can be written to, AND/OR
2 - waagent.conf is designed to be an addition (i.e. customizable extension) to a (for security reasons non-writeable) Datasource,

with Datasource referring to the file used by cloud-init.

Is option 2 achievable? And how can I test that?

Finally, note that the introduction of cloud-init version 0.7.6 in the Ubuntu images would be desirable: this version of cloud-init works fine in Trusty (14.04.1 LTS) and also has the option to create swap with (advanced) cloud-config settings.

Can this new cloud-init version be introduced?

Kind regards.....

[Scott Moser (smoser)]

In cloud-init right now we have the following inputs of configuration data, in increasing order of precedence.

a.) system configuration in /etc/cloud/cloud.cfg
b.) system configuration in /etc/cloud/cloud.cfg.d/*.cfg
    these are increasing in C locale sorted order (thus they use XX-name.cfg , XX being a number, for more obvious sorting).
c.) datasource
d.) vendor-data
e.) user-data

As it is, there is no way then to override the 'disk_setup' and 'fs_setup' configuration that azure datasource provides.

One path to this would be to change the azure datasource to read:
 Datasource:
   Azure:
    cloud_config:
      disk_setup: ...
      fs_setup: ...

Then the datasource check to see if there were values in the passed-in ds_cfg, and use those instead of its BUILTIN_CLOUD_CONFIG if there were.

That could work, but without further thinking I dont know oif it would suffice. The datasource object is generally pretty static , and cloud-init might not re-read it on second boot (after the user had configured the new settings).

[trialotto (trialotto)]

@Scott (and everyone else),

It has been a while since visiting this launchpad and/or since doing some decent testing.

Nevertheless, I recall correctly, there is this option (also discussed on this launchpad) to work with partitions and disk setups on Azure VMs: just inject some custom data.

The custom data option can be used to define cloud-config settings and hence partitions and so on.

Daniel Watkins has created a Wiki page, based upon the topic thread somewhere in this launchpad.

The page can be found at: https://wiki.ubuntu.com/AzureSwapPartitions

However, note that the information provided on that page is not really up to date and, moreover, it sometimes does not work as expected and/or the instructions are not entirely correct.

Just view the page as some introduction.

@Scott, with respect to these lines

"One path to this would be to change the azure datasource to read:
 Datasource:
   Azure:
    cloud_config:
      disk_setup: ...
      fs_setup: ...

Then the datasource check to see if there were values in the passed-in ds_cfg, and use those instead of its BUILTIN_CLOUD_CONFIG if there were.

That could work, but without further thinking I dont know oif it would suffice. The datasource object is generally pretty static , and cloud-init might not re-read it on second boot (after the user had configured the new settings)."

one can state that

- it is not strictly necessary to use the "Datasource" stanza, (and)

- cloud-config settings, injected via custom data, will suffice (with some minor exceptions), (and)

- issues can still occur on Azure VMs, since there is this nasty habit of creating a NTFS volume at stop/start sequences, which habit is entirely related to some lines of odd code in the relevant Azure scripts (and I am pretty sure that a patch has not been created until now)

In general, read all of the above with some caution, it has been a while ago for me.

Regards......

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2484