neutron-openvswitch-agent seems to create wrong OVS flows for newly created networks. This causes package losses, including lost DHCP requests, resulting in instances that did not receive an IP. This can cause tempest tests to fail.
Restarting the neutron-openvswitch-agent will result in properly created OVS flows, and the traffic flowing properly.
The issue has been observed using VLAN as a network_type and no other special configurations (like DVR being enabled), while running tempest tests: tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.
For example, testr run tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops randomly fails due to the fact that the instance did not get any reply for its DHCP request. NOTE: while the OVS flows are wrong for each run of the test, the test still has a small chance of success.
Hypervisor used: Hyper-V. The port on Hyper-V are properly bound and the VLAN tag properly set, the ports are marked as "alive". The network traffic arrives with the proper VLAN tag at the Networking Controller. The bug is unrelated to neutron-hyperv-agent and Hyper-V.
This issue has been observed in the Liberty release.
Details: http://paste.openstack.org/show/476764/
IRC discussion: http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2015-10-19.log.html#t2015-10-19T20:27:31
For a brief summary. The local VLAN mapping between a network VLAN and the br-int tag seems to be getting mangled. Somehow two tags become associated with one real external network. This leads to incorrect rewrite rules for the devices on the other tag.