3) The systemd-networkd files generated by netplan differ for interfaces that have IP addresses configures vs the ones that do not (.network file is not generated for interfaces without an IP address)
Before reboot the interface was up (as a result of a charm doing ip link up enp4s0f1):
$ ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7ae7:d1ff:fe24:d731/64 scope link
valid_lft forever preferred_lft forever
After reboot the interface is down:
ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
And systemd-networkd reports enp4s0f1 as "off" and unmanaged:
sudo networkctl list | grep enp4s0
IDX LINK TYPE OPERATIONAL SETUP
2 enp4s0f0 ether routable configured
3 enp4s0f1 ether off unmanaged
Adding a network file for enp4s0f1 and restarting systemd-networkd fixes the problem:
ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7ae7:d1ff:fe24:d731/64 scope link
valid_lft forever preferred_lft forever
Maybe not adding the network section sets it to "degraded" in networkd but at least the link is set to "up".
Netplan should handle this case, otherwise it is not possible to properly reboot OpenStack network node without manual steps to restore cloud datapath.
Subscribed field-high (netplan) as this impacts the ability of network nodes in OpenStack deployments to come up with a working data-plane.
Example impact:
1) an ovs bridge uses an interface enp4s0f1 for a data-port (Neutron + OVS datapath) 0.0.1:6633"
is_ connected: true
Interface phy-br-data
type: patch
options: {peer=int-br-data}
Interface "enp4s0f1"
ovs-vsctl show
Bridge br-data
Controller "tcp:127.
fail_mode: secure
Port phy-br-data
Port "enp4s0f1"
2) netplan contains the following configuration enp4s0f1
network:
macaddress: 78:e7:d1:24:d7:31
set- name: enp4s0f1
addresses:
gateway4: 10.232.0.1
macaddress: 78:e7:d1:24:d7:30
nameserver s:
addresses:
search:
set- name: enp4s0f0
version: 2
ethernets:
enp4s0f1:
match:
mtu: 1500
enp4s0f0:
- 10.232.1.139/21
match:
mtu: 1500
- 10.232.0.2
- maas
- newmaas
3) The systemd-networkd files generated by netplan differ for interfaces that have IP addresses configures vs the ones that do not (.network file is not generated for interfaces without an IP address)
cat /run/systemd/ network/ 10-netplan- enp4s0f1* 78:e7:d1: 24:d7:31
[Match]
MACAddress=
[Link]
Name=enp4s0f1
WakeOnLan=off
MTUBytes=1500
tree /run/systemd/ network/ network/ enp4s0f0. 2728.netdev enp4s0f0. 2728.network enp4s0f0. 2730.netdev enp4s0f0. 2730.network enp4s0f0. 2731.netdev enp4s0f0. 2731.network enp4s0f0. 2732.netdev enp4s0f0. 2732.network enp4s0f0. link enp4s0f0. network enp4s0f1. link enp5s0d1. link enp5s0. link
/run/systemd/
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
├── 10-netplan-
└── 10-netplan-
Before reboot the interface was up (as a result of a charm doing ip link up enp4s0f1):
$ ip a s enp4s0f1 MULTICAST, PROMISC, UP,LOWER_ UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000 d1ff:fe24: d731/64 scope link
3: enp4s0f1: <BROADCAST,
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7ae7:
valid_lft forever preferred_lft forever
After reboot the interface is down:
ip a s enp4s0f1 MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000
3: enp4s0f1: <BROADCAST,
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
And systemd-networkd reports enp4s0f1 as "off" and unmanaged:
sudo networkctl list | grep enp4s0
IDX LINK TYPE OPERATIONAL SETUP
2 enp4s0f0 ether routable configured
3 enp4s0f1 ether off unmanaged
Adding a network file for enp4s0f1 and restarting systemd-networkd fixes the problem:
cat > /run/systemd/ network/ 10-netplan- enp4s0f1. network 78:e7:d1: 24:d7:31
[Match]
MACAddress=
Name=enp4s0f1
sudo systemctl restart systemd-networkd
ip a s enp4s0f1 MULTICAST, UP,LOWER_ UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000 d1ff:fe24: d731/64 scope link
3: enp4s0f1: <BROADCAST,
link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
inet6 fe80::7ae7:
valid_lft forever preferred_lft forever
sudo networkctl list | grep enp4s0
2 enp4s0f0 ether routable configured
3 enp4s0f1 ether degraded configured
Maybe not adding the network section sets it to "degraded" in networkd but at least the link is set to "up".
Netplan should handle this case, otherwise it is not possible to properly reboot OpenStack network node without manual steps to restore cloud datapath.