Comment 21 for bug 1864822

@Slawek - I was able to reproduce this behavior in Devstack (Master) configured for DVR. For your convenience, I have provided the output from "stock" devstack (legacy routers) and DVR-enabled devstack (dvr_snat).

NOTE: This seems to impact 'vlan' networks, NOT 'flat' networks, in a DVR scenario.

-=-=-=-=-=-

== STOCK ==

local.conf:

[[local|localrc]]
ADMIN_PASSWORD=password
DATABASE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
Q_PLUGIN=ml2
Q_ML2_TENANT_NETWORK_TYPE=vxlan
disable_service horizon cinder swift

DevStack Version: ussuri
Change: 01826e1c5b65e8d9c88b4f195bb688137b28c0c5 Merge "Remove fixup_virtualenv" 2020-04-09 16:00:35 +0000
OS Version: CentOS 7.7.1908 Core

2020-04-11 22:23:12.009 | stack.sh completed in 1034 seconds.
[jdenton@localhost devstack]$

[jdenton@localhost ~]$ cat /etc/neutron/l3_agent.ini | grep agent_mode
#agent_mode = legacy

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x511d1a5fb4dc2c8, duration=213.398s, table=0, n_packets=31, n_bytes=2098, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x511d1a5fb4dc2c8, duration=248.562s, table=0, n_packets=41, n_bytes=3810, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x511d1a5fb4dc2c8, duration=248.580s, table=0, n_packets=12, n_bytes=1272, priority=0 actions=NORMAL

source openrc admin admin
openstack network create --provider-network-type vlan --provider-physical-network public --provider-segment 1000 vlan1000
openstack subnet create --subnet-range 192.168.77.0/24 --network vlan1000 subnet1000

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x511d1a5fb4dc2c8, duration=250.059s, table=0, n_packets=31, n_bytes=2098, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x511d1a5fb4dc2c8, duration=1.864s, table=0, n_packets=2, n_bytes=180, priority=4,in_port="phy-br-ex",dl_vlan=4 actions=mod_vlan_vid:1000,NORMAL
 cookie=0x511d1a5fb4dc2c8, duration=285.223s, table=0, n_packets=45, n_bytes=4194, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x511d1a5fb4dc2c8, duration=285.241s, table=0, n_packets=12, n_bytes=1272, priority=0 actions=NORMAL

>> Simulate disconnect by restarting OVS:
sudo systemctl restart openvswitch

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x4957ef95c18218f4, duration=4.505s, table=0, n_packets=0, n_bytes=0, priority=4,in_port="phy-br-ex",dl_vlan=4 actions=mod_vlan_vid:1000,NORMAL
 cookie=0x4957ef95c18218f4, duration=4.477s, table=0, n_packets=0, n_bytes=0, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x4957ef95c18218f4, duration=4.545s, table=0, n_packets=0, n_bytes=0, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x4957ef95c18218f4, duration=4.550s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL

>> Everything looks OK.

-=-=-=-=-=-=-

== DVR ==

local.conf:

[[local|localrc]]
ADMIN_PASSWORD=password
DATABASE_PASSWORD=password
RABBIT_PASSWORD=password
SERVICE_PASSWORD=password
Q_PLUGIN=ml2
Q_ML2_TENANT_NETWORK_TYPE=vxlan
Q_DVR_MODE=dvr_snat
disable_service horizon cinder swift

DevStack Version: ussuri
Change: 01826e1c5b65e8d9c88b4f195bb688137b28c0c5 Merge "Remove fixup_virtualenv" 2020-04-09 16:00:35 +0000
OS Version: CentOS 7.6.1810 Core

2020-04-11 22:51:07.324 | stack.sh completed in 1088 seconds.

[jdenton@localhost devstack]$ cat /etc/neutron/l3_agent.ini | grep agent_mode
agent_mode = dvr_snat
#agent_mode = legacy

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x9e09489cfebc5585, duration=278.338s, table=0, n_packets=25, n_bytes=1774, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x9e09489cfebc5585, duration=317.281s, table=0, n_packets=77, n_bytes=7066, priority=2,in_port="phy-br-ex" actions=resubmit(,1)
 cookie=0x9e09489cfebc5585, duration=317.604s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL
 cookie=0x9e09489cfebc5585, duration=317.278s, table=0, n_packets=14, n_bytes=1412, priority=1 actions=resubmit(,3)
 cookie=0x9e09489cfebc5585, duration=317.277s, table=1, n_packets=77, n_bytes=7066, priority=0 actions=resubmit(,2)
 cookie=0x9e09489cfebc5585, duration=317.275s, table=2, n_packets=77, n_bytes=7066, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x9e09489cfebc5585, duration=317.274s, table=3, n_packets=14, n_bytes=1412, priority=1 actions=NORMAL

>> Unlike legacy mode, we do not have a 'drop' flow in Table 0 from the get-go. That seems to move to table 2 in this scenario.

source openrc admin admin
openstack network create --provider-network-type vlan --provider-physical-network public --provider-segment 1000 vlan1000
openstack subnet create --subnet-range 192.168.77.0/24 --network vlan1000 subnet1000

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x9e09489cfebc5585, duration=313.117s, table=0, n_packets=25, n_bytes=1774, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x9e09489cfebc5585, duration=352.060s, table=0, n_packets=81, n_bytes=7422, priority=2,in_port="phy-br-ex" actions=resubmit(,1)
 cookie=0x9e09489cfebc5585, duration=352.383s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL
 cookie=0x9e09489cfebc5585, duration=352.057s, table=0, n_packets=14, n_bytes=1412, priority=1 actions=resubmit(,3)
 cookie=0x9e09489cfebc5585, duration=352.056s, table=1, n_packets=81, n_bytes=7422, priority=0 actions=resubmit(,2)
 cookie=0x9e09489cfebc5585, duration=0.586s, table=2, n_packets=1, n_bytes=90, priority=4,in_port="phy-br-ex",dl_vlan=4 actions=mod_vlan_vid:1000,NORMAL
 cookie=0x9e09489cfebc5585, duration=352.054s, table=2, n_packets=80, n_bytes=7332, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x9e09489cfebc5585, duration=352.053s, table=3, n_packets=14, n_bytes=1412, priority=1 actions=NORMAL

>> Simulate disconnect by restarting OVS:
sudo systemctl restart openvswitch

[jdenton@localhost devstack]$ sudo ovs-ofctl dump-flows br-ex
 cookie=0x7390a2b2dfa57b5, duration=3.487s, table=0, n_packets=0, n_bytes=0, priority=4,in_port="phy-br-ex",dl_vlan=2 actions=strip_vlan,NORMAL
 cookie=0x7390a2b2dfa57b5, duration=3.835s, table=0, n_packets=0, n_bytes=0, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x9e09489cfebc5585, duration=4.078s, table=0, n_packets=0, n_bytes=0, priority=1 actions=resubmit(,3)
 cookie=0x7390a2b2dfa57b5, duration=3.838s, table=0, n_packets=0, n_bytes=0, priority=0 actions=NORMAL
 cookie=0x9e09489cfebc5585, duration=4.077s, table=1, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,2)
 cookie=0x7390a2b2dfa57b5, duration=3.584s, table=2, n_packets=0, n_bytes=0, priority=4,in_port="phy-br-ex",dl_vlan=4 actions=mod_vlan_vid:1000,NORMAL
 cookie=0x9e09489cfebc5585, duration=4.076s, table=2, n_packets=0, n_bytes=0, priority=2,in_port="phy-br-ex" actions=drop
 cookie=0x9e09489cfebc5585, duration=4.075s, table=3, n_packets=0, n_bytes=0, priority=1 actions=NORMAL

>> Notice the drop flow in Table 0 now. We also lose the 'resubmit(,1)' flow. Not to mention, some flows have the old cookie.