ovsfw tempest tests random fails

[Slawek Kaplonski]

Some random tests from neutron-tempest-ovsfw job are failing very often. Only common thing in those failing tests is that test fails because instance is not reachable and it looks that it is not reachable because of some missing firewall Openflow rules.

Example of failing tests can be found e.g. on: http://logs.openstack.org/57/542257/4/check/neutron-tempest-ovsfw/cf76bcb/logs/testr_results.html.gz or on http://logs.openstack.org/54/537654/4/check/neutron-tempest-ovsfw/5c90b2b/logs/testr_results.html.gz (except hard_reboot test which is different problem and is covered in https://bugs.launchpad.net/neutron/+bug/1747709)

[Slawek Kaplonski (slaweq)]

From my tests on local environment it looks that problem is with missing openflow rules because when I found instance which wasn't reachable during such test and I restarted neutron-openvswitch-agent process, instance was again reachable as agent recreated all OF rules in br-int.
Here are example rules from br-int from moment when instance (connected to ofport=1689) was not reachable: http://paste.openstack.org/show/666969/
Here are example rules from br-int from moment when instance (connected to ofport=1689) was reachable: http://paste.openstack.org/show/666970/ (it was after neutron-openvswitch-agent restart)

[Slawek Kaplonski (slaweq)]

From couple of tests which I did in DNM patch: https://review.openstack.org/#/c/543100/1 it looks that patch https://review.openstack.org/#/c/542596/5 fixes issue with random failed tests described here.

[IWAMOTO Toshihiro (iwamoto)]

Those two failure examples have ovs-vswitchd core dumps so we may be able to mark this bug as a duplicate of bug/1742863.

[Slawek Kaplonski (slaweq)]

@IWAMOTO: I would personally not mark it as duplicate but rather add two bugs as "closes-bug" to commit message in Your patch https://review.openstack.org/#/c/542596 - what You think about it?

[Slawek Kaplonski (slaweq)]

Patch https://review.openstack.org/#/c/542596/ is merged and it fixes this issue with randomly failed tests