Network bridge issue after updating kernel to 5.4.0-137

[siben17]

Hello,

After updating the kernel from 5.4.0-136 to 5.4.0-137 network bridge are not working any more.
I can see the bridge with "ip add show" or "brctl show". Network is up and I can reach the full network from the host through the bridge.

VM on the host can not reach network or host and host can not reach the VM through the bridge.

We have bonded interface used in the bridge and one bridge without IP and one with an IP. DHCP is off on booth bridge.

Downgrading the kernel to 5.4.136 (Ubuntu 5.4.0-136.153-generic 5.4.218) seem to work fine.

Thank you for your help.

Details
Description: Ubuntu 20.04.5 LTS
Release: 20.04
Not working version => Ubuntu 5.4.0-137.154-generic 5.4.218

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Laurent (syrius-no-log)]

Exact same issue here with jammy and going from 5.15.0-57.63 to 5.15.0-58.64
the issue is also reproduced with 5.15.0.59.57 (proposed)

Context is xen host, bond + vlans + bridge for xen VMs
Network is ok on the host, VM are not working, very weird, you can see some traffic coming through the bridge to the VM, but not on the other way around.

I've not been able to reproduce with any kernel-ppa linux-image-generic from 5.15.75 to 5.15.88

looking at https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?h=v5.15.88&qt=grep&q=netback
it looks like https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/?h=Ubuntu-5.15.0-58.64&id=1e4fa84b47473a57ee7f55ed06fc1df5d349a086 is not enough

[Jon Schewe (jpschewe)]

I'm seeing this as well. Some information that may help. I found the problem with 5.4.0-137. I found that when looking at tcpdump on the guest and the bridge I can see ARP requests on both interfaces. On the guest I can see it responding to the ARP request, however I don't see the ARP response on the bridge interface. I booted back to 5.4.0-126 with Xen and everything is fine.

[Simon Kirby (simon-tagadab)]

Also seeing on Jammy 5.15.0.58.56 with a similar setup (xen, lacp bond, bridges using openvswitch). Initially tried downgrading vswitch to 2.17.0 from 2.17.3 which didn't make any difference. Both versions of vswitch are working on the original kernel 5.15.0-25 as well as 5.15.0-33.

[Semen Stepanov (k14412)]

xen:
version : #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022
machine : x86_64
Ubuntu 22.04.1 LTS

Update kernel from 5.15.0-56-generic to 5.15.0-60-generic = paravirtual vms in bridge dont see athers
return kernel to 5.15.0-60-generic = all vm ok

guest vm Ubuntu 20.04.5 LTS kernel update from 5.4.0-133-generic to 5.4.0-139-generic = network not working
guest vm return kernel 5.4.0-133-generic = works fine

# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.c684fea1dd70 yes eno1
       vif1.0

[Andy Howell (andy-jjcftv6wldnzq84cs)]

Ubuntu 20.04.5 LTS
xen-hypervisor-4.11-amd64 4.11.3+24-g14b62ab3e5-1ubuntu2.3

Reverted back to kernel 5.4.0-135 and bridging with Xen VMs is working again.

Spent the day trying to work out why bridging suddenly stopped working. Guest is 18.04.6 LTS.

I'm pretty sure that under 137, the path cost was 100 for enp6s0f0, instead of the 19 it shows now.

brctl showstp xenbr1
xenbr1
 bridge id 8000.90e2ba065da8
 designated root 8000.90e2ba065da8
 root port 0 path cost 0
 max age 20.00 bridge max age 20.00
 hello time 2.00 bridge hello time 2.00
 forward delay 15.00 bridge forward delay 15.00
 ageing time 300.00
 hello timer 0.00 tcn timer 0.00
 topology change timer 0.00 gc timer 230.64
 flags

enp6s0f0 (1)
 port id 8001 state forwarding
 designated root 8000.90e2ba065da8 path cost 19
 designated bridge 8000.90e2ba065da8 message age timer 0.00
 designated port 8001 forward delay timer 0.00
 designated cost 0 hold timer 0.00
 flags

vif1.0 (2)
 port id 8002 state forwarding
 designated root 8000.90e2ba065da8 path cost 100
 designated bridge 8000.90e2ba065da8 message age timer 0.00
 designated port 8002 forward delay timer 0.00
 designated cost 0 hold timer 0.00
 flags

[Wolfgang Karall-Ahlborn (biz+launchpad)]

I can confirm for Ubuntu 22.04, but also affects Ubuntu 18.04 (all using a dummy bridge for Xen domUs in my case).

-> last working:
22.04:
  - linux-image-5.15.0-57-generic
18.04:
  - linux-image-4.15.0-201-generic

-> broken:
22.04:
  - linux-image-5.15.0-58-generic
  - linux-image-5.15.0-60-generic
18.04:
  - linux-image-4.15.0-202-generic
  - linux-image-4.15.0-204-generic

[Stephan Küffner (skueffner)]

On Ubuntu 20.04 with new released Kernel 5.4.0-139 also not working.
Guest 18.04 connected over Bridge on Host.

Currently I stay on kernel < 5.4.0-137 and everything works fine.

[Lolo (8-l)]

looks like the missing cherry picked xen/netback patches have only been merged 13 days ago.
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/log/?h=master-next&qt=grep&q=netback

vs
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?h=v5.15.88&qt=grep&q=netback

[Wolfgang Karall-Ahlborn (biz+launchpad)]

The latest updates seem to have fixed this.

-> working again:
22.04:
  - linux-image-5.15.0-67-generic
18.04:
  - linux-image-4.15.0-206-generic

[siben17 (siben17)]

Confirm working again :

20.04
  - linux-image-5.4.0-144-generic