network not configured when ipv6 netbooted into cloud-init

[LaMont Jones]

https://bugs.launchpad.net/ubuntu/+source/klibc/+bug/1621507 talks of how IPv6 netboot with iscsi root disk doesn't work, blocking IPv6-only MAAS.

After I hand-walked busybox through getting an IPv6 address, everything worked just fine until cloud-init couldn't fetch the instance data, because it insisted on bringing up the interface in IPv4, and there is no IPv4 DHCP on that vlan.

Please work with initramfs and friends on getting IPv6 netboot to actually configure the interface. This may be as simple as teaching it about "inet6 dhcp" interfaces, and bolting the pieces together. Note that "use radvd" is not really an option for our use case.

Related bugs:
 * bug 1621507: initramfs-tools configure_networking() fails to dhcp IPv6 addresses
 * bug 1635716: Can't bring up a machine on a dual network (ipv4 and ipv6)
 * bug 1639930: initramfs network configuration ignored if only ip6= on kernel command line

[Impact]

It is not possible to enlist, commmission, or deploy with MAAS in an IPv6-only environment. Anyone wanting to netboot with a network root filesystem in an IPv6-only environment is affected.

This upload addresses this by accepting, using, and forwarding any IPV6* variables from the initramfs boot. (See https://launchpad.net/bugs/1621507)

[Test Case]

See Bug 1229458. Configure radvd, dhcpd, and tftpd for your IPv6-only netbooting world. Pass the boot process an IPv6 address to fetch instance-data from, and see it fail to configure the network.

[Regression Potential]

1) If the booting host is in a dual-boot environment, and the instance-dat URL uses a hostname that has both A and AAAA RRsets, the booting host may try to talk IPv6 to get instance data. If the instance-data providing host is only allowing that to happen over IPv4, it will fail. (It also represents a configuraiton issue on the providing host...)

[LaMont Jones (lamont)]

With the attached patch applied to0.7.8-1-g3705bb5-0ubuntu1~16.04.1, we still fail.

Details of the failure are attached below.

[LaMont Jones (lamont)]

/run/net-eno1.conf from the booted system. The various packages needed to reproduce this are found in ppa:lamont/maas.

[LaMont Jones (lamont)]

In the category of "highly related":

grep -v ^# /etc/network/interfaces.d/50-cloud-init.cfg
auto lo
iface lo inet loopback

iface eno1 inet dhcp
    dns-nameservers 2001:67c:1562:8010::2:1
    netmask 64

[Ryan Harper (raharper)]

When curtin deploys a machine, it emits a disable-networking config; I suspect that the cloud-init upgrade/change/fix may have removed it as the 50-cloud-init.cfg is only written out if networking is not disabled.

curtin does the following:

echo "network: {config: disabled}" > etc/cloud/cloud.cfg.d/curtin-disable-cloudinit-networking.cfg

[Steve Langasek (vorlon)]

Hello LaMont, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-1-g3705bb5-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.8-8-g0439d8a-0ubuntu1

---------------
cloud-init (0.7.8-8-g0439d8a-0ubuntu1) yakkety; urgency=medium

  * New upstream snapshot.
    - Decode unicode types in decode_binary [Robert Schweikert]
    - systemd: Ensure that cloud-init-local happens before NetworkManager
    - Allow ephemeral drive to be unpartitioned [Paul Meyer]
    - subp: add 'update_env' argument

 -- Scott Moser <email address hidden> Fri, 23 Sep 2016 16:51:27 -0400

[Brian Murray (brian-murray)]

As far as I can tell this doesn't seem to be fixed in cloud-initramfs-tools for Yakkety. Am I missing something?

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.29ubuntu2

---------------
cloud-initramfs-tools (0.29ubuntu2) yakkety; urgency=medium

  * Add IPV6 support to cloud-initramfs-dyn-netconf. (LP: #1621615)

 -- LaMont Jones <email address hidden> Tue, 27 Sep 2016 16:07:48 -0600

[Stéphane Graber (stgraber)]

Hello LaMont, or anyone else affected,

Accepted cloud-initramfs-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.27ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello LaMont, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-1-g3705bb5-0ubuntu1~16.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Steve Langasek (vorlon)]

Hello LaMont, or anyone else affected,

Accepted cloud-initramfs-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.27ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[LaMont Jones (lamont)]

The packages have been verified in xenial-proposed.

[Martin Pitt (pitti)]

The verification of the Stable Release Update for cloud-initramfs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.8-1-g3705bb5-0ubuntu1~16.04.3

---------------
cloud-init (0.7.8-1-g3705bb5-0ubuntu1~16.04.3) xenial-proposed; urgency=medium

  * ntp: move to run after apt configuration (LP: #1628337).

cloud-init (0.7.8-1-g3705bb5-0ubuntu1~16.04.2) xenial; urgency=medium

  * Support IPv6 config coming from initramfs. LP: #1621615.

 -- Scott Moser <email address hidden> Mon, 03 Oct 2016 12:22:26 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.27ubuntu1.2

---------------
cloud-initramfs-tools (0.27ubuntu1.2) xenial; urgency=medium

  * Add IPV6 support to cloud-initramfs-dyn-netconf. (LP: #1621615)

 -- LaMont Jones <email address hidden> Tue, 04 Oct 2016 14:52:53 -0600

[LaMont Jones (lamont)]

My currently-pending MPs address the issue and work in conjunction with the new fixes for Bug#1621507:

https://code.launchpad.net/~lamont/cloud-init/+git/bug-1621615-device6/+merge/309718
https://code.launchpad.net/~lamont/cloud-initramfs-tools/bug-1621615-device6/+merge/309719

These need to land in zesty, and get SRUed to xenial and yakkety.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.31ubuntu1

---------------
cloud-initramfs-tools (0.31ubuntu1) zesty; urgency=medium

  * cloud-initramfs-dyn-netconf: Further adjustments to support
    IPV6 and the new DEVICE6 and net6-DEVICE.conf files.
    (LP: #1621615)

 -- Scott Moser <email address hidden> Wed, 02 Nov 2016 15:55:39 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.8-35-gc24187e-0ubuntu1

---------------
cloud-init (0.7.8-35-gc24187e-0ubuntu1) zesty; urgency=medium

  * New upstream snapshot.
    - pyflakes: fix issue with pyflakes 1.3 found in ubuntu zesty-proposed.

 -- Scott Moser <email address hidden> Mon, 07 Nov 2016 13:31:30 -0500

[Steve Langasek (vorlon)]

Hello LaMont, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-47-gb6561a1-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Steve Langasek (vorlon)]

Hello LaMont, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Scott Moser (smoser)]

I'm marking this verification-done for xenial and cloud-init.
This is based on the following:
a.) the updated code for this bug is only used when ip= is on the command line
b.) an update to initramfs-tools is needed (and not yet present) in order to easily test
c.) the code path is tested functional and identical in zesty

I'd like to suggest that this code is fixed for this bug, and further shortcomings in the implementation will be handled in other bugs.

[Scott Moser (smoser)]

I marked 'verification-done-xenial-cloud-init' dont know what the right way to say "this is tested for cloud-init" is.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.04.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - Add activate_datasource, for datasource specific code paths.
      (LP: #1611074)
    - systemd: cloud-init-local use RequiresMountsFor=/var/lib/cloud
      (LP: #1642062)

cloud-init (0.7.8-47-gb6561a1-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
      + add Before=NetworkManager.service so that cloud-init can render
        NetworkManager network config before it would apply them.
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - lxd: Update network config for LXD 2.3 [Stéphane Graber] (LP: #1640556)
    - Decode unicode types in decode_binary [Robert Schweikert]
    - Allow ephemeral drive to be unpartitioned [Paul Meyer]
    - subp: add 'update_env' argument which allows for more easily adding
      environment variables to a subprocess call.
    - Adjust mounts and disk configuration for systemd. (LP: #1611074)
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + DigitalOcean: use meta-data for network configuration and
        enable data source by default. [Ben Howard]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
     ...

[Chris Halse Rogers (raof)]

Hello LaMont, or anyone else affected,

Accepted cloud-initramfs-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.27ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello LaMont, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chris Halse Rogers (raof)]

Hello LaMont, or anyone else affected,

Accepted cloud-initramfs-tools into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.30ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[LaMont Jones (lamont)]

Discussed in IRC, all of the test cases in question have been met.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.27ubuntu1.3

---------------
cloud-initramfs-tools (0.27ubuntu1.3) xenial; urgency=medium

  * sync with upstream at 0.32ubuntu1 (revno 129)
    * overlayroot: fix overlayroot=crypt with newer initramfs-tools
      (LP: #1634310)
    * overlayroot: support random seed from systemd systemd-random-seed.service
    * cloud-initramfs-dyn-netconf: Further adjustments to support
      IPV6 and the new DEVICE6 and net6-DEVICE.conf files. (LP: #1621615)
    * overlayroot: support 'overlay' filesystem explicitly rather than
      relying on ubuntu specific kernel module 'overlayfs'. (LP: #1493188)
    * overlayroot: write debug to /run/initramfs not /dev/.initramfs but
      support writing to /dev/.initramfs if that is all there is. (LP: #1485752)
    * overlayroot: fix overlayroot if recurse=0 was not provided.
      This fuctionality was lost since workdir support was added. (LP: #1619459)
    * whitespace cleanup.

 -- Scott Moser <email address hidden> Mon, 28 Nov 2016 20:12:51 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.10.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
      + fixed example to not overwrite /etc/hosts [Chris Glass]
      + fix spelling / typos in ca_certs and scripts_vendor.
      + improve HACKING.rst file
      + Add documentation for logging features. [Wesley Wiedenmeier]
    - code style and unit test changes:
      + pep8: fix style errors reported by pycodestyle 2.1.0
      + pyflakes: fix issue with pyflakes 1.3 found in ubuntu zesty-proposed.
      + Add coverage dependency to bddeb to fix package build.
      + Add coverage collection to tox unit tests. [Joshua Powers]
      + do not read system /etc/cloud/cloud.cfg.d (LP: #1635350)
      + tests: silence the Cheetah UserWarning about NameMapper C version.
      + Fix python2.6 things found running in centos 6.

 -- Scott Moser <email address hidden> Tue, 22 Nov 2016 17:04:36 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.30ubuntu1.1

---------------
cloud-initramfs-tools (0.30ubuntu1.1) yakkety; urgency=medium

  * sync with upstream at 0.32ubuntu1 (revno 129)
    * overlayroot: fix overlayroot=crypt with newer initramfs-tools
      (LP: #1634310)
    * overlayroot: support random seed from systemd
      systemd-random-seed.service
    * cloud-initramfs-dyn-netconf: Further adjustments to support
      IPV6 and the new DEVICE6 and net6-DEVICE.conf files.
      (LP: #1621615)

 -- Scott Moser <email address hidden> Mon, 28 Nov 2016 20:21:14 -0500

[Scott Moser (smoser)]

This is fixed in cloud-init 0.7.9.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2724