ovsfw: OVS br-int rule disappears from the table=60 after stop/start VM

[Anton Kurbatov]

I found out that after VM creation and after VM stop/start the set of OVS rules is different in br-int table=60 (TRANSIENT_TABLE)

I have a flat network, in this network I create a VM. After the VM stop/start the set of rules in table 60 for this VM is different from the one that was after VM creation.

Here is a demo:

[root@devstack0 ~]# openstack server create test-vm --image cirros-0.6.2-x86_64-disk --network public --flavor m1.tiny -c id
+-------+--------------------------------------+
| Field | Value |
+-------+--------------------------------------+
| id | 84c7ed9c-c78e-4d15-8a09-6eb18b0f872a |
+-------+--------------------------------------+
[root@devstack0 ~]# openstack port list --device-id 84c7ed9c-c78e-4d15-8a09-6eb18b0f872a -c ID -c mac_address
+--------------------------------------+-------------------+
| ID | MAC Address |
+--------------------------------------+-------------------+
| 4fd0022b-223d-43ac-9134-1623b38ee2a6 | fa:16:3e:4b:db:3e |
+--------------------------------------+-------------------+
[root@devstack0 ~]#

Table 60: two rules with dl_dst=fa:16:3e:4b:db:3e after VM is created:

[root@devstack0 neutron]# ovs-ofctl dump-flows br-int table=60 | grep fa:16:3e:4b:db:3e
 cookie=0x1a51dc2aa3392248, duration=23.420s, table=60, n_packets=0, n_bytes=0, idle_age=1961, priority=90,vlan_tci=0x0000/0x1fff,dl_dst=fa:16:3e:4b:db:3e actions=load:0x1c->NXM_NX_REG5[],load:0x2->NXM_NX_REG6[],resubmit(,81)
 cookie=0x1a51dc2aa3392248, duration=23.420s, table=60, n_packets=25, n_bytes=2450, idle_age=678, priority=90,dl_vlan=2,dl_dst=fa:16:3e:4b:db:3e actions=load:0x1c->NXM_NX_REG5[],load:0x2->NXM_NX_REG6[],strip_vlan,resubmit(,81)
[root@devstack0 neutron]#

Stop/start the VM and check it again:

[root@devstack0 ~]# openstack server stop test-vm
[root@devstack0 ~]# openstack server start test-vm
[root@devstack0 ~]#
[root@devstack0 neutron]# ovs-ofctl dump-flows br-int table=60 | grep fa:16:3e:4b:db:3e
 cookie=0x1a51dc2aa3392248, duration=14.201s, table=60, n_packets=25, n_bytes=2450, idle_age=697, priority=90,dl_vlan=2,dl_dst=fa:16:3e:4b:db:3e actions=load:0x1d->NXM_NX_REG5[],load:0x2->NXM_NX_REG6[],strip_vlan,resubmit(,81)
[root@devstack0 neutron]#

You can see that the rule [1] has disappeared.

And there is a neutron-openvsiwth-agent message 'Initializing port <port_id> that was already initialized' while VM starting:

Oct 10 08:50:05 devstack0 neutron-openvswitch-agent[232791]: INFO neutron.agent.securitygroups_rpc [None req-df876af2-5007-42ae-ae4e-8c968f59fb5c None None] Preparing filters for devices {'4fd0022b-223d-43ac-9134-1623b38ee2a6'}
Oct 10 08:50:05 devstack0 neutron-openvswitch-agent[232791]: INFO neutron.agent.linux.openvswitch_firewall.firewall [None req-df876af2-5007-42ae-ae4e-8c968f59fb5c None None] Initializing port 4fd0022b-223d-43ac-9134-1623b38ee2a6 that was already initialized.

I get this behavior on devstack with neutron from master branch.

It looks like this rule is disappeared because OVS interface under OVS port is recreated after VM stop/start and new OFPort object is creating with network_type=None (as well with physical_network=None). Compare to a few lines above where the OFPort object is created with network_type/physical_network [2]

I actually discovered this behavior while testing my neutron port-check plugin [3]

[root@devstack0 ~]# openstack port check 4fd0022b-223d-43ac-9134-1623b38ee2a6 -c firewall
+----------+------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------+------------------------------------------------------------------------------------------------------------------------------------------+
| firewall | - No flow: table=60, priority=90,vlan_tci=(0, 8191),eth_dst=fa:16:3e:4b:db:3e actions=set_field:29->reg5,set_field:2->reg6,resubmit(,81) |
+----------+------------------------------------------------------------------------------------------------------------------------------------------+
[root@devstack0 ~]#

[1] https://opendev.org/openstack/neutron/src/commit/78027da56ccb25d19ac2c3bc1c174acb2150e6a5/neutron/agent/linux/openvswitch_firewall/firewall.py#L915
[2] https://opendev.org/openstack/neutron/src/commit/78027da56ccb25d19ac2c3bc1c174acb2150e6a5/neutron/agent/linux/openvswitch_firewall/firewall.py#L724
[3] https://github.com/antonkurbatov/neutron-portcheck

[LIU Yulong (dragon889)]

Seems we meet 2 problems:
1. according to [1], for flat network should not install flow like this:
 cookie=0x1a51dc2aa3392248, duration=14.201s, table=60, n_packets=25, n_bytes=2450, idle_age=697, priority=90,dl_vlan=2,dl_dst=fa:16:3e:4b:db:3e actions=load:0x1d->NXM_NX_REG5[],load:0x2->NXM_NX_REG6[],strip_vlan,resubmit(,81)

for flat network, the vlan matching should be "vlan_tci=0x0000/0x1fff".

2. flow missing after port change ofport:
cookie=0x1a51dc2aa3392248, duration=23.420s, table=60, n_packets=0, n_bytes=0, idle_age=1961, priority=90,vlan_tci=0x0000/0x1fff,dl_dst=fa:16:3e:4b:db:3e actions=load:0x1c->NXM_NX_REG5[],load:0x2->NXM_NX_REG6[],resubmit(,81)

[1] https://opendev.org/openstack/neutron/src/commit/78027da56ccb25d19ac2c3bc1c174acb2150e6a5/neutron/agent/linux/openvswitch_firewall/firewall.py#L910