Comment 0 for bug 1989686

Revision history for this message
Terje Røsten (terjeros) wrote : cloud-init sets wrong netmask causing borken network config on Oracle Cloud

Testing U22.10 Cloud image:

 https://cloud-images.ubuntu.com/kinetic/20220914/kinetic-server-cloudimg-amd64.img

on Oracle Cloud, with with cloud-init 22.3-13-g70ce6442-0ubuntu1~22.10.1.

cloud-init is able to find correct network information (note: the use if /21 as netmask):

ephemeral.py[DEBUG]: Attempting setup of ephemeral network on ens3 with 100.103.27.215/21 brd 100.103.31.255
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'addr', 'add', '100.103.27.215/21', 'broadcast', '100.103.31.255', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'link', 'set', 'dev', 'ens3', 'up'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'append', '0.0.0.0/0', 'via', '100.103.24.1', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'append', '169.254.0.0/16', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
url_helper.py[DEBUG]: [0/3] open 'http://169.254.169.254/opc/v2/instance/' with {'url': 'http://169.254.169.254/opc/v2/instance/', 'stream': False

ephemeral.py[DEBUG]: Attempting setup of ephemeral network on ens3 with 100.103.27.215/21 brd 100.103.31.255
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'addr', 'add', '100.103.27.215/21', 'broadcast', '100.103.31.255', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'link', 'set', 'dev', 'ens3', 'up'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'append', '0.0.0.0/0', 'via', '100.103.24.1', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'append', '169.254.0.0/16', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
url_helper.py[DEBUG]: [0/3] open 'http://169.254.169.254/opc/v2/instance/' with {'url': 'http://169.254.169.254/opc/v2/instance/', 'stream': False, 'allow_redirects': True, 'method': 'GET', 'headers': {'User-Agent': 'Cloud-Init/22.3-13-g70ce6442-0ubuntu1~22.10.1', 'Authorization': 'Bearer Oracle'}} configuration
url_helper.py[DEBUG]: Read from http://169.254.169.254/opc/v2/instance/ (200, 4154b) after 1 attempts
url_helper.py[DEBUG]: [0/3] open 'http://169.254.169.254/opc/v2/vnics/' with {'url': 'http://169.254.169.254/opc/v2/vnics/', 'stream': False, 'allow_redirects': True, 'method': 'GET', 'headers': {'User-Agent': 'Cloud-Init/22.3-13-g70ce6442-0ubuntu1~22.10.1', 'Authorization': 'Bearer Oracle'}} configuration
url_helper.py[DEBUG]: Read from http://169.254.169.254/opc/v2/vnics/ (200, 280b) after 1 attempts
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'del', '169.254.0.0/16', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-4', 'route', 'del', '0.0.0.0/0', 'via', '100.103.24.1', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'link', 'set', 'dev', 'ens3', 'down'] with allowed return codes [0] (shell=False, capture=True)
subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'addr', 'del', '100.103.27.215/21', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True)

However, at later stage it forgets about the netmask:

stages.py[DEBUG]: applying net config names for {'config': [{'name': 'ens3', 'type': 'physical', 'mac_address': '02:00:17:06:ae:e9', 'mtu': 9000, 'subnets': [{'type': 'static', 'address': '100.103.27.215'}]}], 'version': 1}

creates:

$ cat /run/systemd/network/10-netplan-ens3.*
[Match]
MACAddress=02:00:17:06:ae:e9

[Link]
Name=ens3
WakeOnLan=off
MTUBytes=9000
[Match]
MACAddress=02:00:17:06:ae:e9
Name=ens3

[Link]
MTUBytes=9000

[Network]
LinkLocalAddressing=ipv6
Address=100.103.27.215/24

Note /24 here.

If one is unlucky and get an IP in upper octets (in the subnet),
ip command will refuse to set default gateway in routing table as IP of gw is outside subnet.

Hence, we end up with no working network configuration:

root@v:# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:00:17:06:ae:e9 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 100.103.27.215/24 brd 100.103.27.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::17ff:fe06:aee9/64 scope link
       valid_lft forever preferred_lft forever
root@v:# ip r
100.103.27.0/24 dev ens3 proto kernel scope link src 100.103.27.215
root@v:#

Any ideas what's causing this or how to debug more deeply?

Thanks in advance.