Incorrect tag in other-config for openvsiwtch agent after upgrade to mitaka

[George Shuklin]

We've performed upgrade juno->kilo->libery->mitaka (one by one) without rebooting compute hosts.

After mitaka upgrage we found that some of tenant networks are not functional. Deeper debug shows that in openvswitch tag value in 'other-config' field in ovs port description does not match actual tag on the port. (tag field).

This cause openvswitch-agent to set wrong segmentation_id on irrelevant host-local tags.

Visual symptom: after restarting neutron-openvswitch-agent connectivity with given port appears for some time, than disappears. Tcdpump on the physical interface shows, that traffic coming to host with proper segmentation_id, but instance's replies are send back with wrong segmentation_id, which belongs to some random network of the different tenant.

There are two ways to fix this:
1. reboot host
2. write tag field to the tag value of the port and restart neutron-openvswitch-agent.

Example of the incorrectly filled port (ovs-vsctl port list):

_uuid : a5bfb91f-78de-4916-b16a-6ea737cf3b6d
bond_active_slave : []
bond_downdelay : 0
bond_fake_iface : false
bond_mode : []
bond_updelay : 0
external_ids : {}
fake_bridge : false
interfaces : [7fb9c7a6-963c-4814-b9a4-a23d1a918843]
lacp : []
mac : []
name : "tap20802dee-34"
other_config : {net_uuid="9a1923c8-a07d-487e-a96e-310103acd911", network_type=vlan, physical_network=local, segmentation_id="3035", tag="201"}
qos : []
statistics : {}
status : {}
tag : 302
trunks : []
vlan_mode : []

This problems repeated in the few installations of openstack, therefore is not a random fluke.

This script [1] fixes bad tags, but I believe this is a rather serious issue with openvswitch-agent persistency.

[1] https://gist.github.com/amarao/fba1e766cfa217b0342d0fe066aeedd7

Affected version: mitaka, but I believe it related to previous versions, which was: juno, upgraded to kilo, upgraded to liberty, upgraded to mitaka.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Fixed in master, Newton and Mitaka (https://review.openstack.org/#/c/422026/)

This bug is solved.