Xenial Azure: Make generation of network config from IMDS hotplug scripts configurable opt-in

[Chad Smith]

=== Begin SRU Template ===
[Impact]
By default, Xenial needs to rely on existing cloud image hotplug scripts and only generate fallback network config (dhcp on eth0) by default. If consumers want to generate dynamic network from Azure's IMDS service, thus removing cloud image hotplug scripts, then a datasource configuration option is surfaced.

[Test Case]
1. Deploy stock Xenial cloud image
2. upgrade cloud-init -proposed
3. Run cloud-init clean --reboot --logs
4. Confirm that network is not sourced from IMDS content and hotplug scripts still exist
5. Add datasource configuration setting Azure: apply_network_config: true
6. Run cloud-init clean --reboot --logs
7. Confirm that network is sourced from IMDS and hotplug scripts are removed.

[Regression Potential]

[Other Info]
Upstream commit at
  https://git.launchpad.net/cloud-init/commit/?id=15a75ea1

=== End SRU Template ===

=== Original Description ===

cloud-init v. 18.4-0ubuntu1~16.04.1 in -proposed automatically renders network configuration from Azure's IMDS by default instead of fallback config of dhcp on eth0. This represents a difference in behavior from current Xenial.

On Xenial Azure, Ubuntu cloud images have udev scripts to handle network hotplug. Azure datasource has the ability to read full network config from their IMDS service and render hotplugged devices as well as remove the cloud-image default scripts.

Make the cloud-init hotplug behavior configurable and default it to off in Xenial.

[Server Team CI bot (server-team-bot)]

This bug is fixed with commit 1d5e9aef to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=1d5e9aef

[Steve Langasek (vorlon)]

Hello Chad, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/18.4-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Dimitri John Ledkov (xnox)]

this should have had ubuntu task added..... Has this been verified yet?

[Chad Smith (chad.smith)]

Manual validation of IMDS network config off/on config toggle

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 18.4-0ubuntu1~16.04.2

---------------
cloud-init (18.4-0ubuntu1~16.04.2) xenial; urgency=medium

  * cherry-pick 1d5e9aef: azure: Add apply_network_config option to
    disable network (LP: #1798424)
  * debian/patches/openstack-no-network-config.patch
    add patch to default Azure apply_network_config to False. Only
    fallback network config on eth0 is generated by cloud-init. IMDS
    network_config is ignored.

cloud-init (18.4-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * drop the following cherry-picks now included:
    + cpick-3cee0bf8-oracle-fix-detect_openstack-to-report-True-on
  * refresh patches:
   + debian/patches/azure-use-walinux-agent.patch
   + debian/patches/openstack-no-network-config.patch
  * refresh patches:
   + debian/patches/ds-identify-behavior-xenial.patch
  * New upstream release. (LP: #1795953)
    - release 18.4
    - tests: allow skipping an entire cloud_test without running.
    - tests: disable lxd tests on cosmic
    - cii-tests: use unittest2.SkipTest in ntp_chrony due to new deps
    - lxd: adjust to snap installed lxd.
    - docs: surface experimental doc in instance-data.json
    - tests: fix ec2 integration tests. process meta_data instead of meta-data
    - Add support for Infiniband network interfaces (IPoIB). [Mark Goddard]
    - cli: add cloud-init query subcommand to query instance metadata
    - tools/tox-venv: update for new features.
    - pylint: ignore warning assignment-from-no-return for _write_network
    - stages: Fix bug causing datasource to have incorrect sys_cfg.
    - Remove dead-code _write_network distro implementations.
    - net_util: ensure static configs have netmask in translate_network result
      [Thomas Berger]
    - Fall back to root:root on syslog permissions if other options fail.
      [Robert Schweikert]
    - tests: Add mock for util.get_hostname. [Robert Schweikert]
    - ds-identify: doc string cleanup.
    - OpenStack: Support setting mac address on bond. [Fabian Wiesel]
    - bash_completion/cloud-init: fix shell syntax error.
    - EphemeralIPv4Network: Be more explicit when adding default route.
    - OpenStack: support reading of newer versions of metdata.
    - OpenStack: fix bug causing 'latest' version to be used from network.
    - user-data: jinja template to render instance-data.json in cloud-config
    - config: disable ssh access to a configured user account
    - tests: print failed testname instead of docstring upon failure
    - tests: Disallow use of util.subp except for where needed.
    - sysconfig: refactor sysconfig to accept distro specific templates paths
    - Add unit tests for config/cc_ssh.py [Francis Ginther]
    - Fix the built-in cloudinit/tests/helpers:skipIf
    - read-version: enhance error message [Joshua Powers]
    - hyperv_reporting_handler: simplify threaded publisher
    - VMWare: Fix a network config bug in vm with static IPv4 and no gateway.
      [Pengpeng Sun]
    - logging: Add logging config type hyperv for reporting via Azure KVP
      [Andy Liu]
    - tests: disable other snap test as well [Joshua Powers]
    - tests: disable snap, fix write_files binary [Joshua...

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 18.4-22-g6062595b-0ubuntu1

---------------
cloud-init (18.4-22-g6062595b-0ubuntu1) disco; urgency=medium

  * New upstream snapshot.
    - azure: retry imds polling on requests.Timeout (LP: #1800223)
    - azure: Accept variation in error msg from mount for ntfs volumes
      [Jason Zions] (LP: #1799338)
    - azure: fix regression introduced when persisting ephemeral dhcp lease
      [Aswin Rajamannar]
    - azure: add udev rules to create cloud-init Gen2 disk name symlinks
      (LP: #1797480)
    - tests: ec2 mock missing httpretty user-data and instance-identity routes
    - azure: remove /etc/netplan/90-hotplug-azure.yaml when net from IMDS
    - azure: report ready to fabric after reprovision and reduce logging
      [Aswin Rajamannar] (LP: #1799594)
    - query: better error when missing read permission on instance-data
    - instance-data: fallback to instance-data.json if sensitive is absent.
      (LP: #1798189)
    - docs: remove colon from network v1 config example. [Tomer Cohen]
    - Add cloud-id binary to packages for SUSE [Jason Zions]
    - systemd: On SUSE ensure cloud-init.service runs before wicked
      [Robert Schweikert] (LP: #1799709)
    - update detection of openSUSE variants [Robert Schweikert]
    - azure: Add apply_network_config option to disable network from IMDS
      (LP: #1798424)
    - Correct spelling in an error message (udevadm). [Katie McLaughlin]

 -- Chad Smith <email address hidden> Mon, 12 Nov 2018 20:33:12 -0700

[Ryan Harper (raharper)]

This bug is believed to be fixed in cloud-init in version 18.5. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3254