Networking does not work after upgrading libnetplan0

[Lee Mendelowitz]

I upgraded libnetplan0 on our AWS EC2 from 0.99-0ubuntu3~20.04.2 to 0.101-0ubuntu3~20.04.2, but did not upgrade netplan.io, which remained at version 0.99-0ubuntu1. On system reboot, our networking was not configured properly and our AWS EC2 instance was unreachable via SSH.

I believe the issue is that libnetplan 0.101-0ubuntu3~20.04.2 is incompatible with netplan.io 0.99-0ubuntu1. However, "apt show netplan.io=0.99-0ubuntu1" lists the dependency libnetplan0 (>= 0.99-0ubuntu1). I believe this is incorrect, since from this experience libnetplan 0.101-0ubuntu3~20.04.2 appears to be incompatible with netplan.io 0.99-0ubuntu1.

I suspected that the issue was with netplan by inspecting /var/log/syslog, which had these entries after upgrading libnetplan0 - some errors from systemd-networkd and then interface ens5 not coming up:

Feb 18 17:20:44 xxxx systemd-networkd[446]: /run/systemd/network/10-netplan-ens5.network:4: Maximum transfer unit (MTU) value out of range. Permitted range is 68…4294967295, ignoring: -1
Feb 18 17:20:44 xxxx systemd-networkd[446]: /run/systemd/network/10-netplan-ens5.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section.
Feb 18 17:20:44 [omitted] cloud-init[460]: Cloud-init v. 20.4.1-0ubuntu1~20.04.1 running 'init' at Thu, 18 Feb 2021 17:20:44 +0000. Up 5.06 seconds.
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: | ens5 | False | . | . | . | [omitted] |
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: | lo | True | ::1/128 | . | host | . |
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +-------+-------------+---------+-----------+-------+
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: | Route | Destination | Gateway | Interface | Flags |
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +-------+-------------+---------+-----------+-------+
Feb 18 17:20:44 [omitted] cloud-init[460]: ci-info: +-------+-------------+---------+-----------+-------+

I was able to recover the EC2 instance by attaching the root EBS volume to another working EC2 instance and manually installing netplan.io=0.101-0ubuntu3~20.04.2 using chroot. When I rebooted, the server was able to boot with networking.

To reproduce, upgrade libnetplan0 from 0.99-0ubuntu3~20.04.2 to 0.101-0ubuntu3~20.04.2, but keep netplan.io at version version 0.99-0ubuntu1 and reboot.

I'm using apt repositories:
http://us-east-1.ec2.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
http://us-east-1.ec2.archive.ubuntu.com/ubuntu focal/main amd64 Packages

Description: Ubuntu 20.04.2 LTS
Release: 20.04

[Lukas Märdian (slyon)]

It appears there has been some ABI breakage, similar to LP: #1922898

Newer versions of netplan (starting with 0.104) use a strict version dependency to upgrade libnetplan and netplan.io at the same time, to avoid such issues. In addition to being much more careful wrt. ABI breaking changes!