Cloud-init update renders secondary addresses to be incompatible with standard tools

Bug #1675571 reported by Ben Howard on 2017-03-23
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned
curtin
Medium
Unassigned
cloud-init (Ubuntu)
Medium
Unassigned
resolvconf (Debian)
Won't Fix
Unknown
resolvconf (Ubuntu)
Undecided
Unassigned

Bug Description

The change of how cloud-init renders /etc/network/interface.d/50-cloud-init.cfg, standard tools no longer work as expected:
* resolvconf will nullify nameservers
* if* commands ignore secondary addresses

[ORIGINAL REPORT]

Regresion from Bug #1657940.

When provisioning with multiple eth0 addresses, /etc/resolv.conf is empty:

Consider:
root@tester:~# cat /etc/network/interfaces.d/50-cloud-init.cfg
# This file is generated from information provided by
# the datasource. Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 138.197.98.102
    dns-nameservers 8.8.8.8 8.8.4.4
    gateway 138.197.96.1
    netmask 255.255.240.0

# control-alias eth0
iface eth0 inet static
    address 10.17.0.11
    netmask 255.255.0.0

Which then yields an empty /etc/resolv.conf:
root@tester:/run/resolvconf# cat interface/eth0.inet

root@tester:/run/resolvconf# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

The problem is that resolvconfg does pattern matching for eth*.inet. The second definition of eth0 has no nameserver and therefore overrides the definition.

Related branches

description: updated
Ryan Harper (raharper) wrote :

The network config in the previous bug used DHCP on v4 and v6; it appears this instance has a different network config injected. Can you attach the network configuration?

As a workaround, you can include a nameserver configuration

- type: nameserver:
  address:
    - 8.8.8.8
    - 8.8.8.3

Or also include the dns-nameservers attribute in the second subnet configuration.

This will render dns-nameservers entry under lo
which will get picked up.

Given the above config, it looks like the ifupdown resolveconf hooks are not multi-iface-stanza aware.

Changed in cloud-init:
status: New → Incomplete

The following is the network_config:
'_network_config': {'config': [{'mac_address': 'ca:22:2d:f4:1a:5e',
                                 'name': 'eth0',
                                 'subnets': [{'address': '138.197.98.102',
                                              'control': 'auto',
                                              'dns_nameservers': ['8.8.8.8',
                                                                  '8.8.4.4'],
                                              'gateway': '138.197.96.1',
                                              'netmask': '255.255.240.0',
                                              'type': 'static'},
                                             {'address': '10.17.0.11',
                                              'control': 'auto',
                                              'netmask': '255.255.0.0',
                                              'type': 'static'}],
                                 'type': 'physical'}],
                     'version': 1},

That was extracted from /var/lib/cloud/obj.pkl.

Changing the cloud-metadata won't work in this case, since the the code for the DigitalOcean metadata source is assigning the Nameservers to the primary interface. Also, adding the DNS resolvers to the loopback interfaces doesn't work because the loopback interface is already up before the definition is added.

Changed in cloud-init:
status: Incomplete → New

The patch in comment #4 moves the revolvers to the loopback interface; resolve conf picks it up.

Scott Moser (smoser) on 2017-03-27
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
affects: cloud-init → cloud-init (Ubuntu)
Changed in cloud-init:
status: New → Confirmed

I would like to nominate this bug for the Xenial, Yakkety and Zesty branches as well.

So while my patch for the DigitalOcean datasource fixes this usecase, the rendering of additional addresses actually breaks all the "if*" tools for secondary addresses and even IPv6. Consider:

root@utl-ubuntu-16-04-x64-50307:~# ifquery eth0
address: 138.197.91.79
gateway: 138.197.80.1
netmask: 255.255.240.0
broadcast: 138.197.95.255
root@utl-ubuntu-16-04-x64-50307:~# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 4a:41:c5:1b:8a:a0 brd ff:ff:ff:ff:ff:ff
    inet 138.197.91.79/20 brd 138.197.95.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 10.17.0.6/16 brd 10.17.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2604:a880:800:10::2ced:3001/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::4841:c5ff:fe1b:8aa0/64 scope link
       valid_lft forever preferred_lft forever

Additionally, "ifconfig" no longer shows secondary addresses. Notice that `10.17.0.6/16` in #7 above is not defined:

root@utl-ubuntu-16-04-x64-50307:~# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 4a:41:c5:1b:8a:a0
          inet addr:138.197.91.79 Bcast:138.197.95.255 Mask:255.255.240.0
          inet6 addr: fe80::4841:c5ff:fe1b:8aa0/64 Scope:Link
          inet6 addr: 2604:a880:800:10::2ced:3001/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:844 errors:0 dropped:0 overruns:0 frame:0
          TX packets:653 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:107141 (107.1 KB) TX bytes:91481 (91.4 KB)

summary: - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 with multiple eth0 definitions has no
- resolvers
+ Cloud-init update renders secondary addresses ti be incompatible with
+ standard tools
description: updated

I've withdrawn my PR for the DigitalOcean specific fix after an offline conversation.

Scott Moser (smoser) on 2017-03-28
summary: - Cloud-init update renders secondary addresses ti be incompatible with
+ Cloud-init update renders secondary addresses to be incompatible with
standard tools
description: updated
Scott Moser (smoser) on 2017-03-28
Changed in curtin:
status: New → Confirmed
importance: Undecided → Medium
Changed in cloud-init:
importance: Undecided → Medium
Changed in resolvconf (Debian):
status: Unknown → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.