Activity log for bug #1833671

Date Who What changed Old value New value Message
2019-06-21 08:07:03 Tom Hughes bug added bug
2019-06-21 11:21:18 Dan Streetman bug added subscriber Dan Streetman
2019-06-21 13:18:08 Launchpad Janitor systemd (Ubuntu): status New Confirmed
2019-06-21 13:30:53 Justin Goetz bug added subscriber Justin Goetz
2019-06-27 10:26:18 Tobias Karnat bug added subscriber Tobias Karnat
2019-07-23 20:34:11 Dan Streetman nominated for series Ubuntu Bionic
2019-07-23 20:34:11 Dan Streetman bug task added systemd (Ubuntu Bionic)
2019-07-23 20:34:19 Dan Streetman systemd (Ubuntu): status Confirmed Fix Released
2019-07-23 20:34:45 Dan Streetman systemd (Ubuntu Bionic): assignee Dan Streetman (ddstreet)
2019-07-23 20:34:47 Dan Streetman systemd (Ubuntu Bionic): importance Undecided Medium
2019-07-23 20:34:49 Dan Streetman systemd (Ubuntu Bionic): status New In Progress
2019-07-23 20:42:31 Dan Streetman bug watch added https://github.com/systemd/systemd/issues/10118
2019-07-23 20:42:31 Dan Streetman bug task added systemd
2019-07-23 20:42:49 Dan Streetman tags ddstreet-next systemd
2019-07-23 21:37:57 Dan Streetman description Running systemd-networkd from systemd 237-3ubuntu10.23 on Ubuntu 18.04.2 I have one machine where, every time systemd-networkd restarts (ie every time there is an update to systemd) the bond0 interface stops working. I see both physical interfaces go soft down and then come back again: Jun 21 07:28:24 odin.openstreetmap.org systemd[1]: systemd 237 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SEC Jun 21 07:28:24 odin.openstreetmap.org systemd[1]: Detected architecture x86-64. Jun 21 07:28:24 odin.openstreetmap.org kernel: bond0: link status down for backup interface eno2, disabling it in 200 ms Jun 21 07:28:24 odin.openstreetmap.org kernel: bond0: link status down for active interface eno1, disabling it in 200 ms Jun 21 07:28:24 odin.openstreetmap.org kernel: 8021q: adding VLAN 0 to HW filter on device eno2 Jun 21 07:28:25 odin.openstreetmap.org kernel: 8021q: adding VLAN 0 to HW filter on device eno1 Jun 21 07:28:25 odin.openstreetmap.org kernel: bond0: link status up again after 200 ms for interface eno2 Jun 21 07:28:25 odin.openstreetmap.org kernel: bond0: link status up again after 100 ms for interface eno1 and after that nothing until I stop systemd-networkd, delete the bond interface, and then start systemd-networkd again. On most machines the cycle seems to take a bit longer and the interfaces reach a hard down start before coming back and in that case there seems to be no problem. I think this is likely an instance of this upstream bug: https://github.com/systemd/systemd/issues/10118 which has a fix here: https://github.com/systemd/systemd/pull/10465 [impact] restarting systemd-networkd drops carrier on all bond slaves, temporarily interrupting networking over the bond. [test case] on a bionic system with 2 interfaces that can be put into a bond, create config files such as: root@lp1833671:~# cat /etc/systemd/network/10-bond0.netdev [NetDev] Name=bond0 Kind=bond root@lp1833671:~# cat /etc/systemd/network/20-ens8.network [Match] Name=ens8 [Network] Bond=bond0 root@lp1833671:~# cat /etc/systemd/network/20-ens9.network [Match] Name=ens9 [Network] Bond=bond0 root@lp1833671:~# cat /etc/systemd/network/30-bond0.network [Match] Name=bond0 [Network] Address=1.2.3.4/32 restart networkd, or reboot, and verify the bond is up: root@lp1833671:~# ip a 3: ens8: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 link/ether 42:30:62:cc:36:2b brd ff:ff:ff:ff:ff:ff 4: ens9: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond0 state UP group default qlen 1000 link/ether 42:30:62:cc:36:2b brd ff:ff:ff:ff:ff:ff 5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 42:30:62:cc:36:2b brd ff:ff:ff:ff:ff:ff inet 1.2.3.4/32 scope global bond0 valid_lft forever preferred_lft forever inet6 fe80::4030:62ff:fecc:362b/64 scope link valid_lft forever preferred_lft forever restart networkd and check /var/log/syslog: root@lp1833671:~# systemctl restart systemd-networkd root@lp1833671:~# cat /var/log/syslog ... Jul 23 21:08:07 lp1833671 systemd-networkd[1805]: ens9: Lost carrier Jul 23 21:08:07 lp1833671 systemd-networkd[1805]: ens8: Lost carrier Jul 23 21:08:07 lp1833671 systemd-networkd[1805]: ens9: Gained carrier Jul 23 21:08:07 lp1833671 systemd-networkd[1805]: ens8: Gained carrier [regression potential] this changes how bond slaves are managed, so regressions could affect any configurations using bonding. [other info] the patch is already included in d, and ifupdown manages networking in x, so this is needed only for b. [original description] Running systemd-networkd from systemd 237-3ubuntu10.23 on Ubuntu 18.04.2 I have one machine where, every time systemd-networkd restarts (ie every time there is an update to systemd) the bond0 interface stops working. I see both physical interfaces go soft down and then come back again: Jun 21 07:28:24 odin.openstreetmap.org systemd[1]: systemd 237 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SEC Jun 21 07:28:24 odin.openstreetmap.org systemd[1]: Detected architecture x86-64. Jun 21 07:28:24 odin.openstreetmap.org kernel: bond0: link status down for backup interface eno2, disabling it in 200 ms Jun 21 07:28:24 odin.openstreetmap.org kernel: bond0: link status down for active interface eno1, disabling it in 200 ms Jun 21 07:28:24 odin.openstreetmap.org kernel: 8021q: adding VLAN 0 to HW filter on device eno2 Jun 21 07:28:25 odin.openstreetmap.org kernel: 8021q: adding VLAN 0 to HW filter on device eno1 Jun 21 07:28:25 odin.openstreetmap.org kernel: bond0: link status up again after 200 ms for interface eno2 Jun 21 07:28:25 odin.openstreetmap.org kernel: bond0: link status up again after 100 ms for interface eno1 and after that nothing until I stop systemd-networkd, delete the bond interface, and then start systemd-networkd again. On most machines the cycle seems to take a bit longer and the interfaces reach a hard down start before coming back and in that case there seems to be no problem. I think this is likely an instance of this upstream bug: https://github.com/systemd/systemd/issues/10118 which has a fix here: https://github.com/systemd/systemd/pull/10465
2019-07-25 21:24:28 Łukasz Zemczak systemd (Ubuntu Bionic): status In Progress Fix Committed
2019-07-25 21:24:35 Łukasz Zemczak bug added subscriber Ubuntu Stable Release Updates Team
2019-07-25 21:24:38 Łukasz Zemczak bug added subscriber SRU Verification
2019-07-25 21:25:07 Łukasz Zemczak tags ddstreet-next systemd ddstreet-next systemd verification-needed verification-needed-bionic
2019-07-25 23:04:05 Tom Hughes tags ddstreet-next systemd verification-needed verification-needed-bionic ddstreet-next systemd verification-done-bionic verification-needed
2019-07-27 04:42:40 Mathew Hodson systemd (Ubuntu): importance Undecided Medium
2019-08-06 17:27:08 Dan Streetman tags ddstreet-next systemd verification-done-bionic verification-needed systemd verification-done-bionic verification-needed
2019-08-06 18:36:25 Launchpad Janitor systemd (Ubuntu Bionic): status Fix Committed Fix Released
2019-08-06 18:36:46 Brian Murray removed subscriber Ubuntu Stable Release Updates Team
2019-08-07 10:03:07 Launchpad Janitor merge proposal linked https://code.launchpad.net/~fourdollars/ubuntu/+source/systemd/+git/systemd/+merge/370808