duplicate mac address during config-drive configuration with LXD container on openstack

[James Page]

=== Begin SRU Template ===
[Impact]
When the 'ip_gre' module is loaded, the kernel creates two network
devices 'gre0' and 'gretap0' that appear in all network namespaces.
(For example, if you create an lxd container, and then load the ip_gre
module from outside the container, the container will see 2 new
network devices).
The hardware address of these devices is 00:00:00:00:00 as seen below.
   # ( cd /sys/class/net/ && grep . gre*/address )
   gre0/address:00:00:00:00
   gretap0/address:00:00:00:00:00:00

This "duplicate" mac address caused cloud-init to raise a RuntimeError.

The overall impact is that cloudinit's network rendering code will
not work if the ip_gre module is loaded on the system. That will
happen in some nova-lxd environments, but also anywhere where a user
has loaded that module and is running lxc.

[Test Case]

1.) load a module on your host
    sudo modprobe ip_gre

2.) Launch an instance in lxd.

 $ rel=xenial
 $ name=x1
 $ lxc launch ubuntu-daily:$rel $name

3.) see the stack trace by running 'get_interfaces_by_mac()' in the guest.

 $ lxc exec $name -- \
     python3 -c 'from cloudinit import net; print(net.get_interfaces_by_mac())'

4.) upgrade instance to proposed cloud-init
 $ lxc exec $name -- sh -c '
    mirror=http://archive.ubuntu.com/ubuntu
    echo deb $mirror $(lsb_release -sc)-proposed main |
       tee /etc/apt/sources.list.d/proposed.list
    apt-get update -q
    apt-get install -qy cloud-init'
 $ lxc exec $name -- dpkg-query --show cloud-init

5.) see that get_interfaces_by_mac() no longer stack traces.
 $ lxc exec $name -- \
     python3 -c 'from cloudinit import net; print(net.get_interfaces_by_mac())'

A more complete test case is to load an image into nova-lxd with gre
tunneling loaded on the host, but that is much more involved setup.

[Regression Potential]
Regression potential should be pretty low. We are simply ignoring
network interfaces not named 'lo' that have a mac address of '00:00:00:00:00'

[Other Info]
Upstream commit at
  https://git.launchpad.net/cloud-init/commit/?id=2c0655feb9

=== End SRU Template ===

Whilst testing the changes for nova-lxd to resolve issues with use of config-drive, I tripped over this issue; specifically the networking on a config-drive configured LXD instance never starts due to a duplicate MAC address on the lo and greptap0 devices.

Cloud-init v. 0.7.9 running 'init' at Fri, 19 May 2017 13:41:00 +0000. Up 2.0 seconds.
ci-info: ++++++++++++++++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++++++++++++++++
ci-info: +---------+-------+------------------------------+-----------+-------+-------------------------------------------------+
ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
ci-info: +---------+-------+------------------------------+-----------+-------+-------------------------------------------------+
ci-info: | gretap0 | False | . | . | . | 00:00:00:00:00:00 |
ci-info: | eth0 | True | . | . | . | fa:16:3e:1d:aa:ac |
ci-info: | eth0 | True | fe80::f816:3eff:fe1d:aaac/64 | . | link | fa:16:3e:1d:aa:ac |
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | . |
ci-info: | lo | True | ::1/128 | . | host | . |
ci-info: | gre0 | False | . | . | . | 00-00-00-00-31-36-3a-33-00-00-00-00-00-00-00-00 |
ci-info: +---------+-------+------------------------------+-----------+-------+-------------------------------------------------+
2017-05-19 13:41:01,017 - util.py[WARNING]: failed stage init
failed run of stage init
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 647, in status_wrapper
    ret = functor(name, args)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 365, in main_init
    init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 629, in apply_network_config
    netcfg, src = self._find_networking_config()
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 616, in _find_networking_config
    if self.datasource and hasattr(self.datasource, 'network_config'):
  File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", line 150, in network_config
    self.network_json, known_macs=self.known_macs)
  File "/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 638, in convert_net_json
    known_macs = net.get_interfaces_by_mac()
  File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 403, in get_interfaces_by_mac
    (name, ret[mac], mac))
RuntimeError: duplicate mac found! both 'gretap0' and 'lo' have mac '00:00:00:00:00:00'
------------------------------------------------------------
Cloud-init v. 0.7.9 running 'modules:config' at Fri, 19 May 2017 13:41:02 +0000. Up 4.0 seconds.
Cloud-init v. 0.7.9 running 'modules:final' at Fri, 19 May 2017 13:41:04 +0000. Up 6.0 seconds.
Cloud-init v. 0.7.9 finished at Fri, 19 May 2017 13:41:04 +0000. Datasource DataSourceConfigDrive [net,ver=2][source=/config-drive]. Up 6.0 seconds

Related bugs:
 * bug 1692545: guests have unexpected network devices gretap0 and gre0

[Scott Moser (smoser)]

I'm going to pull this fix to cloud-init, given that we've now filed the upstream nova-lxd bug 1692545 to cover the fact that these gretap devices should not really be there (or at least need a justification).

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

This is fixed in artful:

[ 11:48AM 10022 ] [ bdmurray@impulse:/tmp/cloud-init-0.7.9-153-g16a7302f ]
 $ grep -r "test_duplicates_of_empty_mac" *
tests/unittests/test_net.py: def test_duplicates_of_empty_mac_are_ok(self):

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello James, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-153-g16a7302f-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Chad Smith (chad.smith)]

Validated no RuntimeErrors on xenial, yakkety and zesty:

testing test-xenial -----
{'00:00:00:00:00:00': 'lo', '00:16:3e:c4:83:cc': 'eth0', '00:00:00:00': 'gre0'}
cloud-init 0.7.9-153-g16a7302f-0ubuntu1~16.04.1
testing test-yakkety -----
{'00:16:3e:1a:63:6c': 'eth0', '00:00:00:00:00:00': 'lo', '00:00:00:00': 'gre0'}
cloud-init 0.7.9-153-g16a7302f-0ubuntu1~16.10.1
testing test-zesty -----
{'00:16:3e:7f:76:ee': 'eth0', '00:00:00:00:00:00': 'lo', '00:00:00:00': 'gre0'}
cloud-init 0.7.9-153-g16a7302f-0ubuntu1~17.04.1

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~17.04.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~17.04.1) zesty-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network c...

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.10.1) yakkety-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network...

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-153-g16a7302f-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-153-g16a7302f-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * New upstream snapshot.
    - net: fix reading and rendering addresses in cidr format.
      [Dimitri John Ledkov] (LP: #1689346, #1684349)
    - disk_setup: udev settle before attempting partitioning or fs creation.
      (LP: #1692093)
    - GCE: Update the attribute used to find instance SSH keys.
      [Daniel Watkins] (LP: #1693582)
    - nplan: For bonds, allow dashed or underscore names of keys.
      [Dimitri John Ledkov] (LP: #1690480)
    - tests: python2.6: fix unit tests usage of assertNone and format.
    - tests: update docstring on test_configured_list_with_none
    - fix tools/ds-identify to not write None twice.
    - tox/build: do not package depend on style requirements.
    - tests: ntp: Restructure cc_ntp unit tests. [Chad Smith]
    - flake8: move the pinned version of flake8 up to 3.3.0
    - tests: Apply workaround for snapd bug in test case. [Joshua Powers]
    - RHEL/CentOS: Fix dual stack IPv4/IPv6 configuration. [Andreas Karis]
    - disk_setup: fix several issues with gpt disk partitions. (LP: #1692087)
    - function spelling & docstring update [Joshua Powers]
    - tests: Fix unittest bug in ntp tests. [Joshua Powers]
    - tox: move pylint target to 1.7.1
    - Fix get_interfaces_by_mac for empty macs (LP: #1692028)
    - DigitalOcean: remove routes except for the public interface.
      [Ben Howard] (LP: #1681531.)
    - netplan: pass macaddress, when specified, for vlans
      [Dimitri John Ledkov] (LP: #1690388)
    - doc: various improvements for the docs on cc_users_groups.
      [Felix Dreissig]
    - cc_ntp: write template before installing and add service restart
      [Ryan Harper] (LP: #1645644)
    - tests: fix cloudstack unit tests to avoid accessing
      /var/lib/NetworkManager [Lars Kellogg-Stedman]
    - tests: fix hardcoded path to mkfs.ext4 [Joshua Powers] (LP: #1691517)
    - Actually skip warnings when .skip file is present.
      [Chris Brinker] (LP: #1691551)
    - netplan: fix netplan render_network_state signature.
      [Dimitri John Ledkov] (LP: #1685944)
    - Azure: fix reformatting of ephemeral disks on resize to large types.
      (LP: #1686514)
    - make deb: Add devscripts dependency for make deb.
      Cleanup packages/bddeb. [Chad Smith] (LP: #1685935)
    - openstack: fix log message copy/paste typo in _get_url_settings
      [Lars Kellogg-Stedman]
    - unittests: fix unittests run on centos [Joshua Powers]
    - Improve detection of snappy to include os-release and kernel cmdline.
      (LP: #1689944)
    - Add address to config entry generated by _klibc_to_config_entry.
      [Julien Castets] (LP: #1691135)
    - sysconfig: Raise ValueError when multiple default gateways are present.
      [Chad Smith] (LP: #1687485)
    - FreeBSD: improvements and fixes for use on Azure
      [Hongjiang Zhang] (LP: #1636345)
    - Add unit tests for ds-identify, fix Ec2 bug found.
    - fs_setup: if cmd is specified, use shell interpretation.
      [Paul Meyer] (LP: #1687712)
    - doc: document network ...

[Scott Moser (smoser)]

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2896