Activity log for bug #1818340

Date Who What changed Old value New value Message
2019-03-02 14:55:08 Mark Shuttleworth bug added bug
2019-03-02 14:55:08 Mark Shuttleworth attachment added npln.yml https://bugs.launchpad.net/bugs/1818340/+attachment/5242832/+files/npln.yml
2019-03-04 09:45:27 Sebastien Bacher tags regression-proposed
2019-03-04 09:47:57 Sebastien Bacher bug added subscriber Dan Streetman
2019-03-04 09:48:02 Sebastien Bacher bug added subscriber Dimitri John Ledkov
2019-03-04 09:48:43 Sebastien Bacher systemd (Ubuntu): importance Undecided High
2019-03-04 09:51:11 Sebastien Bacher bug added subscriber Daniel Axtens
2019-03-04 10:28:14 Sebastien Bacher bug added subscriber Sebastien Bacher
2019-03-04 13:56:46 Dan Streetman tags regression-proposed regression-proposed sts
2019-03-04 14:57:49 Dan Streetman nominated for series Ubuntu Disco
2019-03-04 14:57:49 Dan Streetman bug task added systemd (Ubuntu Disco)
2019-03-04 14:57:49 Dan Streetman nominated for series Ubuntu Cosmic
2019-03-04 14:57:49 Dan Streetman bug task added systemd (Ubuntu Cosmic)
2019-03-04 14:57:49 Dan Streetman nominated for series Ubuntu Bionic
2019-03-04 14:57:49 Dan Streetman bug task added systemd (Ubuntu Bionic)
2019-03-04 14:57:56 Dan Streetman systemd (Ubuntu Disco): status New Fix Released
2019-03-04 14:58:00 Dan Streetman systemd (Ubuntu Cosmic): status New In Progress
2019-03-04 14:58:01 Dan Streetman systemd (Ubuntu Bionic): status New In Progress
2019-03-04 14:58:09 Dan Streetman systemd (Ubuntu Cosmic): importance Undecided Critical
2019-03-04 14:58:10 Dan Streetman systemd (Ubuntu Bionic): importance Undecided Critical
2019-03-04 14:58:12 Dan Streetman systemd (Ubuntu Cosmic): assignee Dan Streetman (ddstreet)
2019-03-04 14:58:14 Dan Streetman systemd (Ubuntu Bionic): assignee Dan Streetman (ddstreet)
2019-03-04 14:58:19 Dan Streetman systemd (Ubuntu Cosmic): importance Critical High
2019-03-04 14:58:20 Dan Streetman systemd (Ubuntu Bionic): importance Critical High
2019-03-04 15:15:37 Dan Streetman systemd (Ubuntu Bionic): importance High Critical
2019-03-04 15:15:39 Dan Streetman systemd (Ubuntu Cosmic): importance High Critical
2019-03-04 18:38:24 Dan Streetman description I run a number of servers with -proposed enabled and have seen a bunch of this today: Mar 02 16:20:58 4-ridge-fw1 systemd[1]: systemd-networkd.service: Failed with result 'core-dump'. Mar 02 16:20:58 4-ridge-fw1 systemd[1]: Failed to start Network Service. These machines have numerous bonds, so I suspect that's a factor. So far I have only observed the issue on machines with -proposed enabled so I suspect it is a problem with systemd 237-3ubuntu10.14 Example netplan.yaml attached. [Impact] during restart, systemd-networkd fails an assertion and aborts, leaving the system networking partially (if at all) configured. Further restarts continue to fail. [Test Case] Install a bionic system (cosmic affected also) with only systemd-networkd networking (i.e. uninstall or do not configure netplan). Ensure no networkd conf files are in /run/systemd/network. Stop networkd (sudo systemctl stop systemd-networkd). The interface to test with networkd (e.g. ens3) should have no address assigned and should be down. Create a file similar to below, adjusting for interface name: $ cat /etc/systemd/network/10-netplan-ens3.network [Match] Name=ens3 [Network] Address=192.168.122.68/24 Start networkd: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe6e:8c9f/64 scope link valid_lft forever preferred_lft forever Stop networkd; ens3 should retain its address: ubuntu@lp1818340-b:~$ sudo systemctl stop systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe6e:8c9f/64 scope link valid_lft forever preferred_lft forever Start networkd again; the bug is triggered: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd Job for systemd-networkd.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status systemd-networkd.service" and "journalctl -xe" for details. Alternately, instead of separately stopping and then starting networkd, the failure can be reproduced with just a restart. [Regression Potential] TBD [Other Info] This was introduced by the SRU for bug 1812760; both the new behavior of networkd not removing managed addresses/routes from managed interfaces, as well as the assertion failure bug. This does not fail in disco; I believe additional commit(s) from upstream need to be backported. Original description: --- I run a number of servers with -proposed enabled and have seen a bunch of this today: Mar 02 16:20:58 4-ridge-fw1 systemd[1]: systemd-networkd.service: Failed with result 'core-dump'. Mar 02 16:20:58 4-ridge-fw1 systemd[1]: Failed to start Network Service. These machines have numerous bonds, so I suspect that's a factor. So far I have only observed the issue on machines with -proposed enabled so I suspect it is a problem with systemd 237-3ubuntu10.14 Example netplan.yaml attached.
2019-03-04 21:52:36 Dan Streetman description [Impact] during restart, systemd-networkd fails an assertion and aborts, leaving the system networking partially (if at all) configured. Further restarts continue to fail. [Test Case] Install a bionic system (cosmic affected also) with only systemd-networkd networking (i.e. uninstall or do not configure netplan). Ensure no networkd conf files are in /run/systemd/network. Stop networkd (sudo systemctl stop systemd-networkd). The interface to test with networkd (e.g. ens3) should have no address assigned and should be down. Create a file similar to below, adjusting for interface name: $ cat /etc/systemd/network/10-netplan-ens3.network [Match] Name=ens3 [Network] Address=192.168.122.68/24 Start networkd: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe6e:8c9f/64 scope link valid_lft forever preferred_lft forever Stop networkd; ens3 should retain its address: ubuntu@lp1818340-b:~$ sudo systemctl stop systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe6e:8c9f/64 scope link valid_lft forever preferred_lft forever Start networkd again; the bug is triggered: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd Job for systemd-networkd.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status systemd-networkd.service" and "journalctl -xe" for details. Alternately, instead of separately stopping and then starting networkd, the failure can be reproduced with just a restart. [Regression Potential] TBD [Other Info] This was introduced by the SRU for bug 1812760; both the new behavior of networkd not removing managed addresses/routes from managed interfaces, as well as the assertion failure bug. This does not fail in disco; I believe additional commit(s) from upstream need to be backported. Original description: --- I run a number of servers with -proposed enabled and have seen a bunch of this today: Mar 02 16:20:58 4-ridge-fw1 systemd[1]: systemd-networkd.service: Failed with result 'core-dump'. Mar 02 16:20:58 4-ridge-fw1 systemd[1]: Failed to start Network Service. These machines have numerous bonds, so I suspect that's a factor. So far I have only observed the issue on machines with -proposed enabled so I suspect it is a problem with systemd 237-3ubuntu10.14 Example netplan.yaml attached. [Impact] during restart, systemd-networkd fails an assertion and aborts, leaving the system networking partially (if at all) configured. Further restarts continue to fail. [Test Case] Install a bionic system (cosmic affected also) with only systemd-networkd networking (i.e. uninstall or do not configure netplan). Ensure no networkd conf files are in /run/systemd/network. Stop networkd (sudo systemctl stop systemd-networkd). The interface to test with networkd (e.g. ens3) should have no address assigned and should be down. Create a file similar to below, adjusting for interface name: $ cat /etc/systemd/network/10-netplan-ens3.network [Match] Name=ens3 [Network] Address=192.168.122.68/24 Start networkd: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000     link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff     inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3        valid_lft forever preferred_lft forever     inet6 fe80::5054:ff:fe6e:8c9f/64 scope link        valid_lft forever preferred_lft forever Stop networkd; ens3 should retain its address: ubuntu@lp1818340-b:~$ sudo systemctl stop systemd-networkd ubuntu@lp1818340-b:~$ ip a show ens3 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000     link/ether 52:54:00:6e:8c:9f brd ff:ff:ff:ff:ff:ff     inet 192.168.122.68/24 brd 192.168.122.255 scope global ens3        valid_lft forever preferred_lft forever     inet6 fe80::5054:ff:fe6e:8c9f/64 scope link        valid_lft forever preferred_lft forever Start networkd again; the bug is triggered: ubuntu@lp1818340-b:~$ sudo systemctl start systemd-networkd Job for systemd-networkd.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status systemd-networkd.service" and "journalctl -xe" for details. Alternately, instead of separately stopping and then starting networkd, the failure can be reproduced with just a restart. Note the failure only happens with statically-assigned addresses; interfaces configured with dhcp do not trigger this bug. [Regression Potential] TBD [Other Info] This was introduced by the SRU for bug 1812760; both the new behavior of networkd not removing managed addresses/routes from managed interfaces, as well as the assertion failure bug. This does not fail in disco; I believe additional commit(s) from upstream need to be backported. Original description: --- I run a number of servers with -proposed enabled and have seen a bunch of this today: Mar 02 16:20:58 4-ridge-fw1 systemd[1]: systemd-networkd.service: Failed with result 'core-dump'. Mar 02 16:20:58 4-ridge-fw1 systemd[1]: Failed to start Network Service. These machines have numerous bonds, so I suspect that's a factor. So far I have only observed the issue on machines with -proposed enabled so I suspect it is a problem with systemd 237-3ubuntu10.14 Example netplan.yaml attached.
2019-03-05 12:36:17 Dan Streetman bug added subscriber Ubuntu Stable Release Updates Team
2019-03-05 13:04:00 Daniel Axtens attachment added 0001-Do-not-call-link_enter_set_routes-until-LINK_STATE_S.patch https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1818340/+attachment/5243670/+files/0001-Do-not-call-link_enter_set_routes-until-LINK_STATE_S.patch
2019-03-05 14:50:52 Dominique Poulain bug added subscriber Dominique Poulain
2019-03-05 15:25:27 Dan Streetman systemd (Ubuntu Bionic): status In Progress Fix Released
2019-03-05 15:25:29 Dan Streetman systemd (Ubuntu Cosmic): status In Progress Fix Released