cloud-init fails to run on latest cosmic minimal image

[Dan Watkins]

When cloud-init runs in the latest cosmic minimal image, we see the following traceback:

cloud-init[108]: Traceback (most recent call last):
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 658, in status_wrapper
cloud-init[108]: ret = functor(name, args)
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
cloud-init[108]: init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 648, in apply_network_config
cloud-init[108]: netcfg, src = self._find_networking_config()
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in _find_networking_config
cloud-init[108]: if self.datasource and hasattr(self.datasource, 'network_config'):
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceOpenStack.py", line 115, in network_config
cloud-init[108]: self.network_json, known_macs=None)
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 655, in convert_net_json
cloud-init[108]: known_macs = net.get_interfaces_by_mac()
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 595, in get_interfaces_by_mac
cloud-init[108]: (name, ret[mac], mac))
cloud-init[108]: RuntimeError: duplicate mac found! both 'sit0' and 'tunl0' have mac '00:00:00:00'

It appears that the devices presented by the latest linux-kvm is now different; this is the table output by cloud-init in the previous minimal image:

+++++++++++++++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++++++++++++++
+--------+-------+------------------------------+---------------+--------+-------------------+
| Device | Up | Address | Mask | Scope | Hw-Address |
+--------+-------+------------------------------+---------------+--------+-------------------+
| ens2 | True | 10.220.168.37 | 255.255.255.0 | global | fa:16:3e:ef:89:9a |
| ens2 | True | fe80::f816:3eff:feef:899a/64 | . | link | fa:16:3e:ef:89:9a |
| lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
| lo | True | ::1/128 | . | host | . |
| sit0 | False | . | . | . | . |
+--------+-------+------------------------------+---------------+--------+-------------------+

Whereas the new image shows:

+++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++
+--------+-------+-----------+-----------+-------+-------------------+
| Device | Up | Address | Mask | Scope | Hw-Address |
+--------+-------+-----------+-----------+-------+-------------------+
| ens2 | False | . | . | . | fa:16:3e:15:a7:3b |
| lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
| lo | True | ::1/128 | . | host | . |
| sit0 | False | . | . | . | . |
| tunl0 | False | . | . | . | . |
+--------+-------+-----------+-----------+-------+-------------------+

Specifically, the tunl0 device has been added.

[Dan Watkins (daniel-thewatkins)]

It's not clear to me whether this is a cloud-init bug or a kernel bug, so I've filed it in cloud-images for triage for now.

[Chad Smith (chad.smith)]

Can we get the network configuration passed to this instance or the cloud-init.logs.tar it might shed more light on a configuration that we need to take into account in cloud-init. We can certainly attempt to ignore duplicate mac checks in cloud-init if macs are returned as 6-tuple of 00. But we probably want to make sure we are also understanding the full network config in this case too.

[Chad Smith (chad.smith)]

cloud-init collect-logs would harvest a tarfile of all cloud-init artifacts.

[Server Team CI bot (server-team-bot)]

This bug is fixed with commit 00e36d3d to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=00e36d3d

[panticz.de (panticz.de)]

The Ubuntu 18.04 minimal cloud image from 22-Nov-2018 20:29 (https://cloud-images.ubuntu.com/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img)
is still affected from this issue:

cloud-init[108]: Cloud-init v. 18.4-0ubuntu1~18.04.1 running 'init-local' at Mon, 03 Dec 2018 08:49:26 +0000. Up 18.64 seconds.
cloud-init[108]: 2018-12-03 08:49:27,381 - util.py[WARNING]: failed stage init-local
cloud-init[108]: failed run of stage init-local
cloud-init[108]: ------------------------------------------------------------
cloud-init[108]: Traceback (most recent call last):
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 658, in status_wrapper
cloud-init[108]: ret = functor(name, args)
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
cloud-init[108]: init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 648, in apply_network_config
cloud-init[108]: netcfg, src = self._find_networking_config()
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in _find_networking_config
cloud-init[108]: if self.datasource and hasattr(self.datasource, 'network_config'):
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", line 155, in network_config
cloud-init[108]: self.network_json, known_macs=self.known_macs)
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 655, in convert_net_json
cloud-init[108]: known_macs = net.get_interfaces_by_mac()
cloud-init[108]: File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 595, in get_interfaces_by_mac
cloud-init[108]: (name, ret[mac], mac))
cloud-init[108]: RuntimeError: duplicate mac found! both 'sit0' and 'tunl0' have mac '00:00:00:00'
cloud-init[108]: ------------------------------------------------------------
[[0;1;31mFAILED[0m] Failed to start Initial cloud-init job (pre-networking).