cloud-init does not support ignoring a network interface with NetworkManager

[Thomas Stringer]

NetworkManager will attempt to manage all network interfaces with no configuration. To ignore an interface, you need to specify an ifcfg file in /etc/sysconfig/network-scripts to include "NM_CONTROLLED=no".

Currently, cloud-init does not have support for the NM_CONTROLLED option or any way for a data source to explicitly generate config to tell NetworkManager to ignore a particular interface.

This functionality is currently possible with Netplan, and this report is a request for feature parity in NetworkManager config rendering.

[Dan Watkins (oddbloke)]

Hi Thomas, thanks for the bug report!

Can you give some example network configuration which does what you expect on a netplan-using system, but doesn't on your NM-using system? Some more details about the NM-using system (e.g. distro) would be good too, to enable us to reproduce the issue.

As an aside: netplan does have support for rendering NetworkManager configuration (as well as networkd config, the default), so I think it's more accurate to describe this as an issue with cloud-init's sysconfig renderer.

I'm moving this to Incomplete, please move it back to New once you've provided the above.

Thanks!

Dan

[Thomas Stringer (trstringer)]

Thanks for the quick response, Dan! So with netplan it is an opt-in networking configuration. So what we currently do is look through all available interfaces. Then for all interfaces with IP addresses, we explicitly add them to the desired network configuration.

With netplan that works flawlessly. But with NetworkManager, it is the opposite before. Instead of opt-in, it is opt-out, which means that NetworkManager will pull *all* interfaces unless explicitly removing them from management (by setting NM_CONTROLLED=no in a sysconfig network script).

Currently there is no support in cloud-init to explicitly unmanage an interface with NetworkManager.

The distro is CentOS. And totally understand, this is a sysconfig renderer issue.

Thanks!
Thomas

[Dan Watkins (oddbloke)]

Great, thanks for the background!

I've done a bit more experimentation, and on Ubuntu servers NetworkManager will own and configure devices without specific configuration too, so this is a more general issue. Looking through Netplan's reference doc[0], I can't see any way of disabling this behaviour, so:

> I think it's more accurate to describe this as an issue with cloud-init's sysconfig renderer.

I think it's even more accurate to describe me as wrong. ;)

This feels like something of a grey area: the problem we're hitting is the difference in behaviour between our various backends: NetworkManager is "greedy" by default and will configure any interface that it isn't instructed to ignore, whereas networkd(-via-netplan) is not (and configures only the interfaces for which configuration is provided).

We'll need to loop our netplan folks in to discuss how to introduce these semantics into the netplan configuration format (which is a superset of cloud-init's v2 format), and I'm sure they'll have more background than me.

[0] https://netplan.io/reference/

[Lukas Märdian (slyon)]

Hi Thomas, Dan!

As Dan described I think this boils down to the different defaults in systemd-networkd (configuring only the interfaces it is explicitly instructed to configure) vs NetworkManager (configure all interfaces, which it is not explicitly instructed to ignore).

Generally speaking netplan follows the "only handle explicitly defined config" approach (like systemd-networkd) and does not try to change the defaults of its underlying backends (networkd/NetworkManager/...), as this would probably lead to unexpected behaviour.

Regarding the "NM_CONTROLLED=no" setting: Netplan already writes a config file in "run/NetworkManager/conf.d/netplan.conf", adding the "keyfile.unmanaged-devices+=..." setting (equivalent of "NM_CONTROLLED" in keyfile format), to ignore any interfaces, which are explicitly defined in the netplan YAML config, but not defined to use the NetworkManager render. This does not include interfaces which are not defined at all in the netplan config, though.

So one way to achieve the required outcome, would be to define all interfaces in the netplan config, but select the "networkd" renderer for all interfaces which should not be touched by NetworkManager. Maybe in combination with the new "activation-mode" parameter, to keep those interfaces in a DOWN state: https://github.com/CanonicalLtd/netplan/pull/171 (This new parameter is not yet released, tough, as it is blocked on the systemd v248 release).

[Thomas Stringer (trstringer)]

Thanks for the help and information, Lukas and Dan! We have found a workaround to add "no-auto-default=*" in Network Manager configuration, which provides us with the desired behavior. Chatting with Dan and the upstream team, this work will be prioritized for a more permanent fix.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3837