Fail to set DNS server and search domain when customize Ubuntu18.04 to DHCP IP by cloud-init

[Pengpeng Sun]

The dns servers and search domain was NOT written to /etc/netplan/50-cloud-init.yaml when customize a Ubuntu18.04 Server with DHCP IP and DNS servers/search domain.

From cloud-init.log, I can see it applying network configuration with dns servers and search domain:
2018-06-12 09:50:19,852 - stages.py[INFO]: Applying network configuration from ds bringup=False: {'version': 1, 'config': [{'type': 'physical', 'name': 'ens160', 'mac_address': '00:50:56:bd:45:d2', 'subnets': [{'control': 'auto', 'type': 'dhcp'}]}, {'type': 'nameserver', 'address': ['10.117.21.1', '10.117.21.2'], 'search': ['dns.test.com']}]}

To attach cloud-init logs by 'cloud-init collect-logs' and 50-cloud-init.yaml

Note: If customize the same VM with Static IP and DNS servers/search domain, they will be written to /etc/netplan/50-cloud-init.yaml correctly.

[Pengpeng Sun (pengpengs)]

cloud-init logs

[Pengpeng Sun (pengpengs)]

50-cloud-init.yaml file

[Christian Ehrhardt  (paelzer)]

Hi,
it would be nice to see the full /var/run/vmware-imc/cust.cfg that you are using to configure it.

Just as an FYI - There are plenty of special-case network config sources [1], I see that you fortunately use the generic one, but still on V1. If you code up something new I'd recommend to switch to V2 [2].

Without the actual config from the file requested above I can only derive from what I see in the log, so take it with a grain of salt. With the config made available later that can be checked in detail.

version: 1
config:
  type: physical,
     name: ens160
     mac_address: 00:50:56:bd:45:d2
     subnets:
       - control: auto,
         type: dhcp
  type: nameserver,
     address:
       - 10.117.21.1
       - 10.117.21.2
     search:
       - dns.test.com
}

For now I can't see an obvious flaw, it even sort of matches an example in [3].
The config file requested will help, also I'll help raising it with the cloud-init people.

[1]: http://cloudinit.readthedocs.io/en/latest/topics/network-config.html#network-configuration-sources
[2]: http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html
[3]: http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html

[Ryan Harper (raharper)]

There isn't yet a way to allow DNS overrides and DHCP on the same interface. That is, the DHCP response from the server may include DNS entries so the custom nameserver values are not automatically appended to an existing DHCP configuration.

Generally adding/overriding DHCP response values isn't something that works by default. In 16.04 for example, one needs to customize /etc/dhcp/dhclient.conf or write a dhclient-enter/exit hook.

[Pengpeng Sun (pengpengs)]

Attached the cust.cfg, Thanks Christian.

[Ryan Harper (raharper)]

We're looking to support this use-case in netplan in the near future. I'll update this bug with some details on what changes we'll need to cloud-init to support this use-case.

[Pengzhen(Peter) Cao (pengzhencao)]

This is an important user scenario for VMware customer.
And the fix not only affects ubuntu18.04, but also all other linux distros that are using cloud-init.

The fix needs to be done on a per distro basis, every distro has their specific way to control the behavior if to override dhcp offered dns/search domain. I have the knowledge for how each sepcific OS is doing this and could help with that.

[Graham Leggett (minfrin-y)]

Trying to upgrade from Xenial to Bionic we think we've hit this bug.

Previously in xenial we used resolvconf from bootcmd to override the DNS server. This no longer works in Bionic.

We switched to using the "network" section in cloud init as below:

network:
  version: 1
  config:
    - type: physical
      name: eth0
      subnets:
        - type: dhcp
          dns_nameservers:
            - 172.29.248.2

This config has an effect in xenial but is broken - we get "__init__.py[DEBUG]: no interfaces to rename". No idea why renaming has anything to do with anything, as we're just trying to set a nameserver on an existing interface.

This config in turn has no effect at all in bionic - the DNS server 172.29.248.2 doesn't appear in any logfile.

It seems it's impossible to use bionic as a nameserver, given you cannot override the DHCP provided nameserver to point upstream, and you cannot configure the nameserver because the nameserver is required to download the packages to implement the nameserver.

[Graham Leggett (minfrin-y)]

Also tried the following:

network:
  version: 1
  config:
    - type: nameserver
      address:
        - 172.29.248.2

This version is picked up, but is broken for the same reason xenial is broken:

2018-11-07 14:33:06,581 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/eth0/address
2018-11-07 14:33:06,581 - stages.py[DEBUG]: applying net config names for {'config': [{'address': ['172.29.248.2'], 'type': 'nameserver'}], 'version': 1}
2018-11-07 14:33:06,581 - __init__.py[DEBUG]: no interfaces to rename
2018-11-07 14:33:06,581 - __init__.py[DEBUG]: Datasource DataSourceEc2Local not updated for events: System boot
2018-11-07 14:33:06,581 - stages.py[DEBUG]: No network config applied. Neither a new instance nor datasource network update on 'System boot' event
2018-11-07 14:33:06,581 - main.py[DEBUG]: [local] Exiting. datasource DataSourceEc2Local not in local mode.

[John Savanyo (jsavanyo)]

Can this be prioritized higher as this is a bad regression in >= bionic releases?

[Ryan Harper (raharper)]

Netplan in disco has support for dhcp overrides[A], and this is available in Disco now. Netplan 0.95 is being backported to bionic which will include this functionality.

Once present, for cloud-init; we will need a few things.

1) cloudinit/sources/helpers/vmware/imc/config_nic.py current generates a v1 network config after parsing IMC network configuration. This will need to be updated to indicate the DNSFROMDHCP value in the network config.

2) cloudinit/net/network_state.py will need an update to capture this setting
3) cloudinit/net/{eni,netplan,sysconfig).py will need updates to react to the setting and emit the appropriate config needed to represent the DNSFROMDHCP value.

A. https://github.com/CanonicalLtd/netplan/pull/48/commits/be9dbb43355db4d4576954cdf79df0a9b3c4a219

[Pengzhen(Peter) Cao (pengzhencao)]

I will fix it in cloud-init. Once it is done, we should backport the fix to ubuntu 18.04.1/18.04.2 as their netplan package is being updated.

[Pengzhen(Peter) Cao (pengzhencao)]

We have internal VMware PR2319097 to track the progress.

[Pengzhen(Peter) Cao (pengzhencao)]

Ryan,

Is there any reliable way to detect netplan version in Ubuntu? Could netplan add "-v/--version" option so that cloud-init or VMware own perl customization engine could know which netplan pkg is installed in the system.
We need to check if the Netplan installed could support this "use-dns: true/false" option in the yaml file. I've found there is no "-v/--version" option for netplan and I have to use "dpkg -l" cmd to check the installed netplan version.

[Ryan Harper (raharper)]

Hi,

For now, the version from dpkg is the best way. Netplan does have plans for querying feature-flags/--version.

https://trello.com/c/ls9KUQuV/42-feature-availability-documentation-introspection-feature-flags

[Pengzhen(Peter) Cao (pengzhencao)]

Hi Ryan,

As you said, "cloudinit/sources/helpers/vmware/imc/config_nic.py" now generates v1 network_state dictionary. Should I just add a "DHCPFROMDNS: true/false" key as V1 format or should I update the output to v2 together with other fields?

[Ryan Harper (raharper)]

On Mon, Apr 15, 2019 at 9:21 AM Pengzhen(Peter) Cao <email address hidden>
wrote:

> Hi Ryan,
>
> As you said, "cloudinit/sources/helpers/vmware/imc/config_nic.py" now
> generates v1 network_state dictionary. Should I just add a "DHCPFROMDNS:
> true/false" key as V1 format or should I update the output to v2
> together with other fields?
>

Hi Pengzhen,
I would transition to v2 since that now supports DNSFROMDHCP. I do think
we should add a key in v1, but I'd like to think about what that looks like
before we add it.
For now, both sysconfig (PEERDNS) and netplan (v2) (DNSFROMDHCP) have keys
that
indicate on a per-interface basis whether they should obtain DNS
server/search
from DHCP results.
This will need to be parsed and stored in network_state under the interfaces
dictionary, and then in the renderers, for each interface, if the boolean
is true
emit the correct configuration value (PEERDNS=yes|no, dnsfromdhcp:
true|false).
We can follow up with a change to eni renderer once we determine how to
implement
that under ifupdown/dhclient.

>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1776452
>
> Title:
> Fail to set DNS server and search domain when customize Ubuntu18.04 to
> DHCP IP by cloud-init
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1776452/+subscriptions
>