Comment 22 for bug 1763608

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

Subscribed field-high (netplan) as this impacts the ability of network nodes in OpenStack deployments to come up with a working data-plane.

Example impact:

1) an ovs bridge uses an interface enp4s0f1 for a data-port (Neutron + OVS datapath)
ovs-vsctl show
    Bridge br-data
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port phy-br-data
            Interface phy-br-data
                type: patch
                options: {peer=int-br-data}
        Port "enp4s0f1"
            Interface "enp4s0f1"

2) netplan contains the following configuration enp4s0f1

network:
    version: 2
    ethernets:
        enp4s0f1:
            match:
                macaddress: 78:e7:d1:24:d7:31
            mtu: 1500
            set-name: enp4s0f1
        enp4s0f0:
            addresses:
            - 10.232.1.139/21
            gateway4: 10.232.0.1
            match:
                macaddress: 78:e7:d1:24:d7:30
            mtu: 1500
            nameservers:
                addresses:
                - 10.232.0.2
                search:
                - maas
                - newmaas
            set-name: enp4s0f0

3) The systemd-networkd files generated by netplan differ for interfaces that have IP addresses configures vs the ones that do not (.network file is not generated for interfaces without an IP address)

cat /run/systemd/network/10-netplan-enp4s0f1*
[Match]
MACAddress=78:e7:d1:24:d7:31

[Link]
Name=enp4s0f1
WakeOnLan=off
MTUBytes=1500

tree /run/systemd/network/
/run/systemd/network/
├── 10-netplan-enp4s0f0.2728.netdev
├── 10-netplan-enp4s0f0.2728.network
├── 10-netplan-enp4s0f0.2730.netdev
├── 10-netplan-enp4s0f0.2730.network
├── 10-netplan-enp4s0f0.2731.netdev
├── 10-netplan-enp4s0f0.2731.network
├── 10-netplan-enp4s0f0.2732.netdev
├── 10-netplan-enp4s0f0.2732.network
├── 10-netplan-enp4s0f0.link
├── 10-netplan-enp4s0f0.network
├── 10-netplan-enp4s0f1.link
├── 10-netplan-enp5s0d1.link
└── 10-netplan-enp5s0.link

Before reboot the interface was up (as a result of a charm doing ip link up enp4s0f1):

$ ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7ae7:d1ff:fe24:d731/64 scope link
       valid_lft forever preferred_lft forever

After reboot the interface is down:

ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000
    link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff

And systemd-networkd reports enp4s0f1 as "off" and unmanaged:

sudo networkctl list | grep enp4s0
IDX LINK TYPE OPERATIONAL SETUP
  2 enp4s0f0 ether routable configured
  3 enp4s0f1 ether off unmanaged

Adding a network file for enp4s0f1 and restarting systemd-networkd fixes the problem:

cat > /run/systemd/network/10-netplan-enp4s0f1.network
[Match]
MACAddress=78:e7:d1:24:d7:31
Name=enp4s0f1

sudo systemctl restart systemd-networkd

ip a s enp4s0f1
3: enp4s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 78:e7:d1:24:d7:31 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7ae7:d1ff:fe24:d731/64 scope link
       valid_lft forever preferred_lft forever

sudo networkctl list | grep enp4s0
  2 enp4s0f0 ether routable configured
  3 enp4s0f1 ether degraded configured

Maybe not adding the network section sets it to "degraded" in networkd but at least the link is set to "up".

Netplan should handle this case, otherwise it is not possible to properly reboot OpenStack network node without manual steps to restore cloud datapath.