Activity log for bug #1998749

Date Who What changed Old value New value Message
2022-12-05 07:13:25 LIU Yulong bug added bug
2022-12-05 09:52:10 Slawek Kaplonski tags l3-dvr-backlog
2022-12-05 09:52:15 Slawek Kaplonski neutron: importance Undecided Medium
2022-12-20 03:14:56 OpenStack Infra neutron: status New In Progress
2023-10-19 00:41:53 LIU Yulong description For L3 DVR of VLAN networks, the east west traffic between router subnets will be flooding on the physical bridge. Assuming we have resources like this: 1. subnet-1 10.10.10.0/24 with gateway-1 10.10.10.1, mac-address-[01] 2. subnet-2 20.20.20.0/24 with gateway-2 20.20.20.1, mac-address-[02] 3. subnet-1 and subnet-2 are all connected to router-1 4. host-1 with dvr_host_mac-[03] 5. host-2 with dvr_host_mac-[04] 6. vm-1 on host-1 with IP 10.10.10.100 and mac-address-[05] 7. vm-2 on host-2 with IP 20.20.20.100 and mac-address-[06] The packet procedure looks like this: 1. VM-1 is trying to ping VM-2 2. VM-1 gets the gateway-1's MAC address, and send ping request out. packet tuple (src_mac=[05], des_mac=[01], src_ip=10.10.10.100, dst_ip=20.20.20.100) 3. packet goes to dvr local router namespace, and routing 4. packet source mac address is changed to mac-address-[02] and destination to mac-addres-[06]. packet tuple (src_mac=[02], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) 5. the source MAC address will be changed to dvr_host_mac-[03] on physical bridge (br-vlan), and hit NORMAL packet tuple (src_mac=[01], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) * flooding happens, because the br-vlan had never learnt the destination mac address of VM-2 mac-address-[06]. Tracing the packets, you will see: NORMAL -> no learned MAC for destination, flooding Typically, the flooding action of fast flows are: actions:set(eth(src=fa:16:3f:xx:xx:03)),push_vlan(vid=XXXXX,pcp=0),br-vlan,NIC_PORT 6. On host-2, it hits a dvr_host_mac-[03] flows in table=3 and output to patch-to-int-br, which has a side effect is that the source mac address will not be learnt. packet tuple (src_mac=[01], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) flows: table=3, priority=2,dl_src=fa:16:3f:xx:xx:03 actions=output:"phy-br-vlan" 7. On host-2, VM-2 send reply back, and finally go to br-vlan, it flooding again packet tuple (src_mac=[04], des_mac=[05], src_ip=20.20.20.100, dst_ip=10.10.10.100) 8. ping reply back to host-2, on br-vlan, it hits the output flows which match the dvr_host_mac-[04], then the VM-2's mac address will never be learnt. So, ping request is flooding on br-vlan forever. For L3 DVR of VLAN networks, the east west traffic between router subnets will be flooding on the physical bridge. Assuming we have resources like this: 1. subnet-1 10.10.10.0/24 with gateway-1 10.10.10.1, mac-address-[01] 2. subnet-2 20.20.20.0/24 with gateway-2 20.20.20.1, mac-address-[02] 3. subnet-1 and subnet-2 are all connected to router-1 4. host-1 with dvr_host_mac-[03] 5. host-2 with dvr_host_mac-[04] 6. vm-1 on host-1 with IP 10.10.10.100 and mac-address-[05] 7. vm-2 on host-2 with IP 20.20.20.100 and mac-address-[06] The packet procedure looks like this: 1. VM-1 is trying to ping VM-2 2. VM-1 gets the gateway-1's MAC address, and send ping request out.     packet tuple (src_mac=[05], des_mac=[01], src_ip=10.10.10.100, dst_ip=20.20.20.100) 3. packet goes to dvr local router namespace, and routing 4. packet source mac address is changed to mac-address-[02] and destination to mac-addres-[06].     packet tuple (src_mac=[02], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) 5. the source MAC address will be changed to dvr_host_mac-[03] on physical bridge (br-vlan), and hit NORMAL    packet tuple (src_mac=[03], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) * flooding happens, because the br-vlan had never learnt the destination mac address of VM-2 mac-address-[06].   Tracing the packets, you will see:     NORMAL      -> no learned MAC for destination, flooding   Typically, the flooding action of fast flows are:   actions:set(eth(src=fa:16:3f:xx:xx:03)),push_vlan(vid=XXXXX,pcp=0),br-vlan,NIC_PORT 6. On host-2, it hits a dvr_host_mac-[03] flows in table=3 and output to patch-to-int-br, which has a side effect is that the source mac address will not be learnt. packet tuple (src_mac=[01], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) flows: table=3, priority=2,dl_src=fa:16:3f:xx:xx:03 actions=output:"phy-br-vlan" 7. On host-2, VM-2 send reply back, and finally go to br-vlan, it flooding again packet tuple (src_mac=[04], des_mac=[05], src_ip=20.20.20.100, dst_ip=10.10.10.100) 8. ping reply back to host-2, on br-vlan, it hits the output flows which match the dvr_host_mac-[04], then the VM-2's mac address will never be learnt. So, ping request is flooding on br-vlan forever.
2023-10-19 00:43:17 LIU Yulong description For L3 DVR of VLAN networks, the east west traffic between router subnets will be flooding on the physical bridge. Assuming we have resources like this: 1. subnet-1 10.10.10.0/24 with gateway-1 10.10.10.1, mac-address-[01] 2. subnet-2 20.20.20.0/24 with gateway-2 20.20.20.1, mac-address-[02] 3. subnet-1 and subnet-2 are all connected to router-1 4. host-1 with dvr_host_mac-[03] 5. host-2 with dvr_host_mac-[04] 6. vm-1 on host-1 with IP 10.10.10.100 and mac-address-[05] 7. vm-2 on host-2 with IP 20.20.20.100 and mac-address-[06] The packet procedure looks like this: 1. VM-1 is trying to ping VM-2 2. VM-1 gets the gateway-1's MAC address, and send ping request out.     packet tuple (src_mac=[05], des_mac=[01], src_ip=10.10.10.100, dst_ip=20.20.20.100) 3. packet goes to dvr local router namespace, and routing 4. packet source mac address is changed to mac-address-[02] and destination to mac-addres-[06].     packet tuple (src_mac=[02], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) 5. the source MAC address will be changed to dvr_host_mac-[03] on physical bridge (br-vlan), and hit NORMAL    packet tuple (src_mac=[03], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) * flooding happens, because the br-vlan had never learnt the destination mac address of VM-2 mac-address-[06].   Tracing the packets, you will see:     NORMAL      -> no learned MAC for destination, flooding   Typically, the flooding action of fast flows are:   actions:set(eth(src=fa:16:3f:xx:xx:03)),push_vlan(vid=XXXXX,pcp=0),br-vlan,NIC_PORT 6. On host-2, it hits a dvr_host_mac-[03] flows in table=3 and output to patch-to-int-br, which has a side effect is that the source mac address will not be learnt. packet tuple (src_mac=[01], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) flows: table=3, priority=2,dl_src=fa:16:3f:xx:xx:03 actions=output:"phy-br-vlan" 7. On host-2, VM-2 send reply back, and finally go to br-vlan, it flooding again packet tuple (src_mac=[04], des_mac=[05], src_ip=20.20.20.100, dst_ip=10.10.10.100) 8. ping reply back to host-2, on br-vlan, it hits the output flows which match the dvr_host_mac-[04], then the VM-2's mac address will never be learnt. So, ping request is flooding on br-vlan forever. For L3 DVR of VLAN networks, the east west traffic between router subnets will be flooding on the physical bridge. Assuming we have resources like this: 1. subnet-1 10.10.10.0/24 with gateway-1 10.10.10.1, mac-address-[01] 2. subnet-2 20.20.20.0/24 with gateway-2 20.20.20.1, mac-address-[02] 3. subnet-1 and subnet-2 are all connected to router-1 4. host-1 with dvr_host_mac-[03] 5. host-2 with dvr_host_mac-[04] 6. vm-1 on host-1 with IP 10.10.10.100 and mac-address-[05] 7. vm-2 on host-2 with IP 20.20.20.100 and mac-address-[06] The packet procedure looks like this: 1. VM-1 is trying to ping VM-2 2. VM-1 gets the gateway-1's MAC address, and send ping request out.     packet tuple (src_mac=[05], des_mac=[01], src_ip=10.10.10.100, dst_ip=20.20.20.100) 3. packet goes to dvr local router namespace, and routing 4. packet source mac address is changed to mac-address-[02] and destination to mac-addres-[06].     packet tuple (src_mac=[02], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) 5. the source MAC address will be changed to dvr_host_mac-[03] on physical bridge (br-vlan), and hit NORMAL    packet tuple (src_mac=[03], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) * flooding happens, because the br-vlan had never learnt the destination mac address of VM-2 mac-address-[06].   Tracing the packets, you will see:     NORMAL      -> no learned MAC for destination, flooding   Typically, the flooding action of fast flows are:   actions:set(eth(src=fa:16:3f:xx:xx:03)),push_vlan(vid=XXXXX,pcp=0),br-vlan,NIC_PORT 6. On host-2, it hits a dvr_host_mac-[03] flows in table=3 and output to patch-to-int-br, which has a side effect is that the source mac address will not be learnt. packet tuple (src_mac=[03], des_mac=[06], src_ip=10.10.10.100, dst_ip=20.20.20.100) flows: table=3, priority=2,dl_src=fa:16:3f:xx:xx:03 actions=output:"phy-br-vlan" 7. On host-2, VM-2 send reply back, and finally go to br-vlan, it flooding again packet tuple (src_mac=[04], des_mac=[05], src_ip=20.20.20.100, dst_ip=10.10.10.100) 8. ping reply back to host-2, on br-vlan, it hits the output flows which match the dvr_host_mac-[04], then the VM-2's mac address will never be learnt. So, ping request is flooding on br-vlan forever.