Bogus routes after DHCP lease change

Bug #1831787 reported by Ante Karamatić on 2019-06-05
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
netplan
Undecided
Unassigned
systemd
Unknown
Unknown
systemd (Ubuntu)
High
Dan Streetman
Bionic
Medium
Dan Streetman
Disco
Medium
Dan Streetman
Eoan
High
Dan Streetman

Bug Description

[impact]

networkd does not remove old route(s) after DHCP address change

[test case]

on a system using networkd, that is connected to a network where you can control the addresses that the DHCP server provides, setup system with networkd to get address via DHCP, e.g.

[Match]
Name=ens3

[Network]
DHCP=ipv4

(re)start networkd or reboot, so the system gets an ipv4 DHCP address, and corresponding route to the gateway.

Then on the dhcp server, change the subnet to a different subnet. On the client, once its renews its DHCP address, the server will provide a new address in the new subnet, and the client will add a new default route to the new gateway address. However, the old default route to the old gateway address isn't removed.

Note this also happens without changing the entire subnet, but is more subtle as shown in the original description.

[regression potential]

this affects how networkd handles routes, so has the potential to leave a system with partial or incorrect networking, or no networking at all. Any regression would most likely occur during networkd (re)start or during renewal of a DHCP lease, or when an interface is brought up.

[other info]

original description:
---

Netplan config:

network:
  version: 2
  renderer: networkd
  ethernets:
    eno4:
      dhcp4: no
    eno1np0:
      dhcp4: no
      addresses:
        - 172.16.0.2/24
  bridges:
    br0:
      dhcp4: yes
      interfaces:
        - eno4

On initial boot, machine got 10.0.15.109 IP address:

May 03 13:09:41 ceph2 systemd-networkd[29349]: br0: Configured
May 03 13:09:41 ceph2 systemd-networkd[29349]: br0: DHCPv4 address 10.0.15.109/23 via 10.0.15.253

At one point, DHCP server reserver this IP address and client eventually picked up new IP address:

May 03 15:01:12 ceph2 systemd-networkd[1137]: br0: DHCPv4 address 10.0.15.128/23 via 10.0.15.253

This resulted in IP addresses:

# ip -o a
1: lo inet 127.0.0.1/8 scope host lo\ valid_lft forever preferred_lft forever
1: lo inet6 ::1/128 scope host \ valid_lft forever preferred_lft forever
2: eno1np0 inet 172.16.0.2/24 brd 172.16.0.255 scope global eno1np0\ valid_lft forever preferred_lft forever
2: eno1np0 inet6 fe80::b226:28ff:fe53:56be/64 scope link \ valid_lft forever preferred_lft forever
6: br0 inet 10.0.15.128/23 brd 10.0.15.255 scope global dynamic br0\ valid_lft 503sec preferred_lft 503sec
6: br0 inet6 fe80::b8d7:5eff:fe6b:62a/64 scope link \ valid_lft forever preferred_lft forever

So far, everything is fine. But, the routes on the machine are bogus:

# ip r
default via 10.0.15.253 dev br0 proto dhcp src 10.0.15.109 metric 100
default via 10.0.15.253 dev br0 proto dhcp src 10.0.15.128 metric 100
10.0.14.0/23 dev br0 proto kernel scope link src 10.0.15.128
10.0.15.253 dev br0 proto dhcp scope link src 10.0.15.109 metric 100
10.0.15.253 dev br0 proto dhcp scope link src 10.0.15.128 metric 100
172.16.0.0/24 dev eno1np0 proto kernel scope link src 172.16.0.2

routes with src 10.0.15.109 should have been removed when lease was renewed. I'm not sure if this is a bug in netplan or systemd. This is 18.04, systemd 37-3ubuntu10.21, netplan 0.40.1~18.04.4.

Related branches

Ante Karamatić (ivoks) wrote :

I forgot to mention; this puts machine in a weird state. It can be connected to, via 10.0.15.128, but it can't initiate connections to outside. By removing routes:

# ip r d default via 10.0.15.253 dev br0 proto dhcp src 10.0.15.109 metric 100
# ip r d 10.0.15.253 dev br0 proto dhcp scope link src 10.0.15.109 metric 100

machine is usable again.

Ryan Harper (raharper) wrote :

Can you provide the journal? At least all of:

journalctl -o short-monotonic -u systemd-networkd.service ?

Changed in netplan:
status: New → Incomplete
Ryan Harper (raharper) wrote :

Looks like this issue, I think:

https://github.com/systemd/systemd/issues/12490

Changed in systemd (Ubuntu):
status: New → Incomplete
Ante Karamatić (ivoks) wrote :

It's exactly the same as that systemd issue.

Ryan Harper (raharper) on 2019-06-06
Changed in netplan:
status: Incomplete → Invalid
Changed in systemd (Ubuntu):
importance: Undecided → High
status: Incomplete → Triaged
Dan Streetman (ddstreet) on 2019-07-23
Changed in systemd (Ubuntu):
assignee: nobody → Dan Streetman (ddstreet)
Dan Streetman (ddstreet) on 2019-10-02
Changed in systemd (Ubuntu Eoan):
status: Triaged → In Progress
Changed in systemd (Ubuntu Disco):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Bionic):
assignee: nobody → Dan Streetman (ddstreet)
Changed in systemd (Ubuntu Disco):
importance: Undecided → Medium
Changed in systemd (Ubuntu Bionic):
importance: Undecided → Medium
Dan Streetman (ddstreet) on 2019-10-02
tags: added: bionic disco eoan next-ddstreet systemd
tags: added: ddstreet
removed: next-ddstreet
Dan Streetman (ddstreet) on 2019-10-02
Changed in systemd (Ubuntu Disco):
status: New → In Progress
Changed in systemd (Ubuntu Bionic):
status: New → In Progress
Dan Streetman (ddstreet) on 2019-10-18
description: updated

Hello Ante, or anyone else affected,

Accepted systemd into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemd/242-7ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemd (Ubuntu Eoan):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-eoan

All autopkgtests for the newly accepted systemd (242-7ubuntu3.2) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

gvfs/1.42.1-1ubuntu1 (amd64)
systemd/242-7ubuntu3.2 (ppc64el)
ndctl/unknown (armhf)
casper/1.427 (amd64)
netplan.io/0.98-0ubuntu1 (ppc64el)
munin/unknown (armhf)
linux-oem-osp1/5.0.0-1026.29 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#systemd

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Dan Streetman (ddstreet) wrote :

with dnsmasq server setup:

$ ip -4 a show ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.1/24 scope global ens8
       valid_lft forever preferred_lft forever
$ cat /etc/dnsmasq.d/test

interface=ens8
bind-dynamic

no-resolv
no-poll

domain=test,1.2.3.4/24
dhcp-range=test,1.2.3.100,1.2.3.199,1m
dhcp-host=52:54:00:f7:b2:99,1.2.3.50,1m

on the test system:

ubuntu@lp1831787-e:~$ dpkg -l systemd | grep ii
ii systemd 242-7ubuntu3.2 amd64 system and service manager
ubuntu@lp1831787-e:~$ ip -4 a show dev ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.50/24 brd 1.2.3.255 scope global dynamic ens8
       valid_lft 106sec preferred_lft 106sec
ubuntu@lp1831787-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp src 1.2.3.50 metric 1024
1.2.3.0/24 dev ens8 proto kernel scope link src 1.2.3.50
1.2.3.1 dev ens8 proto dhcp scope link src 1.2.3.50 metric 1024

then on dnsmasq server, change the test system addr to .60:

$ cat /etc/dnsmasq.d/test

interface=ens8
bind-dynamic

no-resolv
no-poll

domain=test,1.2.3.4/24
dhcp-range=test,1.2.3.100,1.2.3.199,1m
dhcp-host=52:54:00:f7:b2:99,1.2.3.60,1m

ubuntu@dhcp-test:~$ sudo systemctl restart dnsmasq

on test system, wait for dhcp lease to timeout:

ubuntu@lp1831787-e:~$ ip -4 a show dev ens8
3: ens8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 1.2.3.60/24 brd 1.2.3.255 scope global dynamic ens8
       valid_lft 114sec preferred_lft 114sec
ubuntu@lp1831787-e:~$ ip -4 r
default via 1.2.3.1 dev ens8 proto dhcp src 1.2.3.60 metric 1024
1.2.3.0/24 dev ens8 proto kernel scope link src 1.2.3.60
1.2.3.1 dev ens8 proto dhcp scope link src 1.2.3.60 metric 1024

tags: added: verification-done verification-done-eoan
removed: verification-needed verification-needed-eoan
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.