[FFe] add support for unique cloud-config templates based on kernel, distro series, arch, python version

[Chad Smith]

Cloud-init allows user-data to provide a "## template: jinja" header line in user-data in order to render custom cloud-config based on any instance metadata found in /run/cloud-init/instance-data.json.

There are a number of use cases where it would be beneficial to provide unique cloud-config user-data based on what ubuntu release, machine architecture kernel or merged_config default_user you an image may have configured.

Allow cloud customers to write a single cloud-config jinja template which can customize configuration options based on what it's operating environment ends up being.

Add 2 top-level keys to the persisted instance-data.json file:

   merged_cfg:
       * The merged cloud-config from /etc/cloud/cloud.cfg and
       /etc/cloud/cloud.cfg.d/*cfg

       * This merged_cfg is helpful in debug and triage of cloud-init bug
       as custom images frequently override Ubuntu certified
       cloud-image defaults.

   sys_info:
       * system platform, arch, kernel and distro info

       * This data is already obtained by cloudinit.util.system_info which
         is used at runtime to determine behavior on every Ubuntu series
         and any other supported distributions.

For ease of use in templates, some of the sys_info fields are generalized as top-level 'v1' standard keys.

The following are the new generalized v1 instance data keys:
    distro, distro_release, distro_version, variant
    kernel_release, system_platform, machine, and python_version

This allows a single #cloud-config user-data which would allow for custom cloud-config based on distro details:

  ## template: jinja
  #cloud-config
   runcmd:
  {% if distro_version == 'xenial' %}
    - echo add custom networking extensions to /etc/network/interfaces.d/
  {% elif distro_version == 'bionic' %}
    - echo add my custom networking extensions to /etc/netplan/
  {% elif distro == 'centos' %}
    - echo do something fun with /etc/sysconfig
  {% endif %}

Potential risk:

 The 'merged_cfg' is sourced from the filesystem in which custom images could place sensitive information such as passwords or keys. That merged_cfg will need to be considered a sensitive_metadata_key that is redacted from the world-readable /run/cloud-init/instance-data.json file.

The branch proposed will need to address this sensitivity and non-root users will not be able to run `cloud-init query merged_cfg` as that data will need to be redacted.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Chad Smith (chad.smith)]

An upstream branch in in review for this feature
https://github.com/canonical/cloud-init/pull/214

[Steve Langasek (vorlon)]

FFe granted.

[Chad Smith (chad.smith)]

Fix released into Ubuntu Focal as of cloud-init 20.1-10-g71af48df-0ubuntu1.

 Expect cloud-images to contain this changeset within the next few days.

The upstream commitish related to this changeset is here
https://github.com/canonical/cloud-init/commit/71af48df3514ca831c90b77dc71ba0a121dec401

Thanks Steve!