netplan.io 0.106.1-7ubuntu0.22.04.2 fails to manage additional loopback addresses on Ubuntu 2204 Jammy

[Gregory Bock]

Attempting to add additional loopback addresses to 22.04 jammy fails with the latest netplan.io package. Previous versions (.104) work correctly as well as newer versions (0.107) in 23.10 mantic. Behavior does not change if default loopback addresses are or are not present in the address list (127.0.0.1/8 and ::1/128).

Netplan is configured via cloudinit in our environment but for simplicity I'll provide output from a manual configuration on a test vm in virtual box.

root@ubuntu-jammy-test:~# cat /etc/netplan/10-loopback.yaml
network:
    version: 2
    ethernets:
        lo:
            addresses:
            - 10.10.10.10/32
            match:
                macaddress: 00:00:00:00:00:00
            set-name: lo
root@ubuntu-jammy-test:~# netplan apply
WARNING:root:Cannot call Open vSwitch: ovsdb-server.service is not running.
root@ubuntu-jammy-test:~# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 enp0s3 ether routable configured

2 links listed.
root@ubuntu-jammy-test:~# apt-get install -y --allow-downgrades netplan.io=0.104-0ubuntu2 libnetplan0=0.104-0ubuntu2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  network-manager | wpasupplicant openvswitch-switch
The following packages will be DOWNGRADED:
  libnetplan0 netplan.io
0 upgraded, 0 newly installed, 2 downgraded, 0 to remove and 0 not upgraded.
Need to get 0 B/181 kB of archives.
After this operation, 163 kB disk space will be freed.
dpkg: warning: downgrading netplan.io from 0.106.1-7ubuntu0.22.04.2 to 0.104-0ubuntu2
(Reading database ... 69845 files and directories currently installed.)
Preparing to unpack .../netplan.io_0.104-0ubuntu2_amd64.deb ...
Unpacking netplan.io (0.104-0ubuntu2) over (0.106.1-7ubuntu0.22.04.2) ...
dpkg: warning: downgrading libnetplan0:amd64 from 0.106.1-7ubuntu0.22.04.2 to 0.104-0ubuntu2
Preparing to unpack .../libnetplan0_0.104-0ubuntu2_amd64.deb ...
Unpacking libnetplan0:amd64 (0.104-0ubuntu2) over (0.106.1-7ubuntu0.22.04.2) ...
Setting up libnetplan0:amd64 (0.104-0ubuntu2) ...
Setting up netplan.io (0.104-0ubuntu2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for dbus (1.12.20-2ubuntu4.1) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
root@ubuntu-jammy-test:~# netplan apply
root@ubuntu-jammy-test:~# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.10.10.10/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback routable configured
  2 enp0s3 ether routable configured

2 links listed.

root@ubuntu-jammy-test:~# apt-get upgrade -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  libnetplan0 netplan.io
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/215 kB of archives.
After this operation, 163 kB of additional disk space will be used.
(Reading database ... 69835 files and directories currently installed.)
Preparing to unpack .../netplan.io_0.106.1-7ubuntu0.22.04.2_amd64.deb ...
Unpacking netplan.io (0.106.1-7ubuntu0.22.04.2) over (0.104-0ubuntu2) ...
Preparing to unpack .../libnetplan0_0.106.1-7ubuntu0.22.04.2_amd64.deb ...
Unpacking libnetplan0:amd64 (0.106.1-7ubuntu0.22.04.2) over (0.104-0ubuntu2) ...
Setting up libnetplan0:amd64 (0.106.1-7ubuntu0.22.04.2) ...
Setting up netplan.io (0.106.1-7ubuntu0.22.04.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for dbus (1.12.20-2ubuntu4.1) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback routable configured
  2 enp0s3 ether routable configured

2 links listed.
root@ubuntu-jammy-test:~# netplan apply
WARNING:root:Cannot call Open vSwitch: ovsdb-server.service is not running.
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback routable unmanaged
  2 enp0s3 ether routable configured

2 links listed.
root@ubuntu-jammy-test:~#

With default addresses after a reboot to reset the environment:

root@ubuntu-jammy-test:~# cat /etc/netplan/10-loopback.yaml
network:
    version: 2
    ethernets:
        lo:
            addresses:
            - 127.0.0.1/8
            - ::1/128
            - 10.10.10.10/32
            match:
                macaddress: 00:00:00:00:00:00
            set-name: lo
root@ubuntu-jammy-test:~# netplan apply
WARNING:root:Cannot call Open vSwitch: ovsdb-server.service is not running.
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback carrier unmanaged
  2 enp0s3 ether routable configured

2 links listed.
root@ubuntu-jammy-test:~# apt-get install -y --allow-downgrades netplan.io=0.104-0ubuntu2 libnetplan0=0.104-0ubuntu2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  network-manager | wpasupplicant openvswitch-switch
The following packages will be DOWNGRADED:
  libnetplan0 netplan.io
0 upgraded, 0 newly installed, 2 downgraded, 0 to remove and 0 not upgraded.
Need to get 0 B/181 kB of archives.
After this operation, 163 kB disk space will be freed.
dpkg: warning: downgrading netplan.io from 0.106.1-7ubuntu0.22.04.2 to 0.104-0ubuntu2
(Reading database ... 69845 files and directories currently installed.)
Preparing to unpack .../netplan.io_0.104-0ubuntu2_amd64.deb ...
Unpacking netplan.io (0.104-0ubuntu2) over (0.106.1-7ubuntu0.22.04.2) ...
dpkg: warning: downgrading libnetplan0:amd64 from 0.106.1-7ubuntu0.22.04.2 to 0.104-0ubuntu2
Preparing to unpack .../libnetplan0_0.104-0ubuntu2_amd64.deb ...
Unpacking libnetplan0:amd64 (0.104-0ubuntu2) over (0.106.1-7ubuntu0.22.04.2) ...
Setting up libnetplan0:amd64 (0.104-0ubuntu2) ...
Setting up netplan.io (0.104-0ubuntu2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for dbus (1.12.20-2ubuntu4.1) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.
root@ubuntu-jammy-test:~# netplan apply
root@ubuntu-jammy-test:~# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 10.10.10.10/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
root@ubuntu-jammy-test:~# networkctl list
IDX LINK TYPE OPERATIONAL SETUP
  1 lo loopback routable configured
  2 enp0s3 ether routable configured

2 links listed.

[Lukas Märdian (slyon)]

This is probably related to the permanentMAC address matching, which was invalid and got dropped in newer version of Netplan, as MAC address matching is not supposed to be used on virtual interfaces (such as the Loopback interface).

Could you please try removing the "match. macaddress: 00:00:00:00:00:00" setting (and "set-name: lo"). That way it should match on the "lo" interface automatically and IP addresses should be assigned.

In the future, you might be able to use "dummy-devices" in Netplan, instead of "lo".

[Gregory Bock (greg-bock)]

If that is the case then why does it work on the newer version that is present in mantic? Is there any source for this package besides github? I can't find any tags, branching, release notes that would make it easy to find/generate a changeset/changelog between specific versions or even find which releases were cut at a certain commit id.

[Gregory Bock (greg-bock)]

Also, why is previous behavior allowed to change in an LTS release?

[Lukas Märdian (slyon)]

That's an interesting find, about it working in Mantic. It needs more investigation.

Previous behavior was a bug that was inconsistent with the NetworkManager backend and the documentation, that's why it got fixed in the LTS, after plenty of testing: https://discourse.ubuntu.com/t/netplan-0-106-call-for-testing/33932

Upstream changelogs can be found here: https://github.com/canonical/netplan/releases
Ubuntu changelogs: https://launchpad.net/ubuntu/+source/netplan.io/+changelog

Tags (packaging repository): https://git.launchpad.net/~ubuntu-core-dev/netplan/+git/ubuntu/refs/

[Gregory Bock (greg-bock)]

I can see the need for skipping most of, if not all non loopback, virtual devices. lo would seem to be a special case imho. I'd like to avoid differing cloudinit nocloud data between the various distros.

As an aside it also doesn't seem to handle subinterfaces (e.g. lo:1) like other distributions do. This is handy in not overwriting the default lo interface settings while setting an alias.

Did someone recently sync github with launchpad? Those release and tags did not appear to be there when I checked weeks ago.

[Gregory Bock (greg-bock)]

Ignore the github comment, I was looking at the https://github.com/canonical/netplan.io repo which I incorrectly assumed was the repo the package was being built from, I didn't even check the code there when I didn't see any versioning tags (facepalm).

[Gregory Bock (greg-bock)]

I was able to confirm that https://github.com/canonical/netplan/pull/278 is the breaking commit with git bisect. I tried to bisect forward to 0.107.1 to see if I could find the behavior change back to working but I think my test setup is flawed or there's another factor at play here. I did spin up a fresh mantic install to verify that it does indeed work. I also verified that cloudinit is generating the same netplan files, and the generated systemd link and network files are also the same between mantic and jammy. I'll attempt to bisect from 0.107.1 back to 0.106.1 on mantic and see what that yields.

[Gregory Bock (greg-bock)]

Mantic with 0.106.1 correctly configures the loopback interface so it seems other package updates on the system aren't affected by the permanent mac address changeset. I suspect it is something with systemd but I haven't found anything definitive yet. The netplan file from cloudinit and the systemd generated network units all appear to have the same content between the OS versions. ethtool doesn't report a permanent address for loopback on jammy or mantic.

[Danilo Egea Gondolfo (danilogondolfo)]

To get you configuration working you need to remove the match and set-name stanzas (which were never required really). Using the definition ID as the interface name (the "lo" inside "ethernets") is enough to configure an interface using its name. And by removing that it will work on Jammy.

As you noticed, there seem to be some change in behavior in newer versions of systemd.

The configuration that uses match.macaddress I wouldn't expect to work actually works on Noble. Although, I've found this by accident:

---
Apr 15 14:14:58 noble (udev-worker)[382]: dummy123: Could not set WakeOnLan to off, ignoring: Operation not supported
Apr 15 14:14:58 noble (udev-worker)[382]: dummy123: Failed to rename network interface 3 from 'dummy123' to 'lo': File exists
Apr 15 14:14:58 noble (udev-worker)[382]: dummy123: Failed to process device, ignoring: File exists
---

I also have a dummy interface in my configuration and it doesn't have a permanent mac address as well. Although, udev apparently tried to apply the set-name from ethernets.lo to this interface as if the match.macaddress matched both.

[Gregory Bock (greg-bock)]

We were using aliased interfaces (lo:1) on other OSs (cent/debian/etc) as well as ubuntu to avoid overwriting the default lo interface configuration files . We've tested cloud-init with lo and no mac matching on ubuntu and non ubuntu OSs and it works, so we'll just change our base CI data to work around this. Anything previous deployed on jammy and working will break with a netplan.io package update which is not ideal, but it is what it is.

Feel free to close this bug.