I was able to duplicate this in neutron version 11.0.0.0rc2.dev368 (OpenStack-Ansible master) under the same conditions. The following log snippet is from neutron-openvswitch-agent.log:


2017-10-31 14:20:24.288 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] ofctl request version=0x4,msg_type=0xe,msg_len=None,xid=0xfda6e61,OFPFlowMod(buffer_id=4294967295,command=0,cookie=1900617720212172640L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],match=OFPMatch(oxm_fields={'arp_tpa': None, 'eth_type': 2054, 'vlan_vid': 4098}),out_group=0,out_port=0,priority=3,table_id=1) timed out: Timeout: 10 seconds
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Error while processing VIF ports: RuntimeError: ofctl request version=0x4,msg_type=0xe,msg_len=None,xid=0xfda6e61,OFPFlowMod(buffer_id=4294967295,command=0,cookie=1900617720212172640L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],match=OFPMatch(oxm_fields={'arp_tpa': None, 'eth_type': 2054, 'vlan_vid': 4098}),out_group=0,out_port=0,priority=3,table_id=1) timed out
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2063, in rpc_loop
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info, ovs_restarted)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/osprofiler/profiler.py", line 157, in wrapper
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1646, in process_network_ports
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     devices_added_updated, ovs_restarted))
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/osprofiler/profiler.py", line 157, in wrapper
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1548, in treat_devices_added_or_updated
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     ovs_restarted)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/osprofiler/profiler.py", line 157, in wrapper
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1431, in treat_vif_port
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     fixed_ips, device_owner, ovs_restarted)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/osprofiler/profiler.py", line 157, in wrapper
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 794, in port_bound
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     device_owner)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/osprofiler/profiler.py", line 157, in wrapper
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     result = f(*args, **kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py", line 568, in bind_port_to_dvr
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     device_owner)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py", line 426, in _bind_distributed_router_interface_port
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     vlan_tag=lvm.vlan, gateway_ip=subnet_info['gateway_ip'])
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_dvr_process.py", line 42, in install_dvr_process_ipv4
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     match=match)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 193, in install_drop
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     instructions=[], match=match, **match_kwargs)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 214, in install_instructions
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self._send_msg(msg)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 95, in _send_msg
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent RuntimeError: ofctl request version=0x4,msg_type=0xe,msg_len=None,xid=0xfda6e61,OFPFlowMod(buffer_id=4294967295,command=0,cookie=1900617720212172640L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],match=OFPMatch(oxm_fields={'arp_tpa': None, 'eth_type': 2054, 'vlan_vid': 4098}),out_group=0,out_port=0,priority=3,table_id=1) timed out
2017-10-31 14:20:24.290 10631 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2017-10-31 14:20:24.293 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Agent rpc_loop - iteration:2 completed. Processed ports statistics: {'regular': {'updated': 0, 'added': 6, 'removed': 0}}. Elapsed:10.127 loop_count_and_wait /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1796
2017-10-31 14:20:24.294 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Loop iteration exceeded interval (5 vs. 10.1274929047)! loop_count_and_wait /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1803
2017-10-31 14:20:24.295 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Agent rpc_loop - iteration:3 started rpc_loop /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1976
2017-10-31 14:20:24.297 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0x4ab14d82,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=12551188769968443967L,duration_nsec=149000000,duration_sec=32,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] _send_msg /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:99
2017-10-31 14:20:24.300 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Agent rpc_loop - iteration:3 - starting polling. Elapsed:0.005 rpc_loop /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2031
2017-10-31 14:20:24.301 10631 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Agent out of sync with plugin!
2017-10-31 14:20:24.302 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): ListPortsCommand(bridge=br-int) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.302 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.303 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbListCommand(if_exists=True, records=[u'qr-b5063eaa-2a', u'sg-4f346d4c-eb', u'qg-ac86fc38-ef', u'fg-3af2714c-ee', u'tap3aa023f7-f9', u'int-br-provider', u'tapce4f293c-d4'], table=Interface, columns=['name', 'external_ids', 'ofport'], row=False) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.305 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.306 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): ListPortsCommand(bridge=br-int) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.307 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.308 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbListCommand(if_exists=True, records=[u'qr-b5063eaa-2a', u'sg-4f346d4c-eb', u'qg-ac86fc38-ef', u'fg-3af2714c-ee', u'tap3aa023f7-f9', u'int-br-provider', u'tapce4f293c-d4'], table=Port, columns=['name', 'tag'], row=False) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.309 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.310 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): ListPortsCommand(bridge=br-int) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.311 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.311 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbListCommand(if_exists=True, records=[u'qr-b5063eaa-2a', u'sg-4f346d4c-eb', u'qg-ac86fc38-ef', u'fg-3af2714c-ee', u'tap3aa023f7-f9', u'int-br-provider', u'tapce4f293c-d4'], table=Interface, columns=['name', 'external_ids', 'ofport'], row=False) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.313 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.314 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Agent rpc_loop - iteration:3 - port information retrieved. Elapsed:0.019 rpc_loop /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2054
2017-10-31 14:20:24.314 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Starting to process devices in:{'current': set([u'ce4f293c-d486-4973-b9b7-29ad9bee7eee', u'3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', u'b5063eaa-2a11-43f7-957d-a3aa5cce4748', u'ac86fc38-efbe-4ca1-86a4-0297e1f358bb', u'4f346d4c-eb4c-42c3-be21-5a86e7cb7097', u'3aa023f7-f998-4502-b699-dfba9a21045c']), 'removed': set([]), 'added': set([u'ce4f293c-d486-4973-b9b7-29ad9bee7eee', u'3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', u'b5063eaa-2a11-43f7-957d-a3aa5cce4748', u'ac86fc38-efbe-4ca1-86a4-0297e1f358bb', u'4f346d4c-eb4c-42c3-be21-5a86e7cb7097', u'3aa023f7-f998-4502-b699-dfba9a21045c'])} rpc_loop /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:2061
2017-10-31 14:20:24.315 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', 'segmentation_id': 201, 'fixed_ips': [{'subnet_id': 'd7a7901f-8452-4c07-be02-6f4cc1521a4c', 'ip_address': '10.10.10.2'}], 'device_owner': u'network:dhcp', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:46:2e:7d', 'device': u'ce4f293c-d486-4973-b9b7-29ad9bee7eee', 'port_security_enabled': False, 'port_id': 'ce4f293c-d486-4973-b9b7-29ad9bee7eee', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.316 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', 'segmentation_id': 201, 'fixed_ips': [{'subnet_id': 'd7a7901f-8452-4c07-be02-6f4cc1521a4c', 'ip_address': '10.10.10.6'}], 'device_owner': u'network:floatingip_agent_gateway', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:93:a8:1a', 'device': u'3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', 'port_security_enabled': False, 'port_id': '3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.317 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e', 'segmentation_id': 200, 'fixed_ips': [{'subnet_id': '840940c3-f766-4dc2-b2e8-8d3eb0ceb20f', 'ip_address': '192.168.1.1'}], 'device_owner': u'network:router_interface_distributed', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:7f:d4:b5', 'device': u'b5063eaa-2a11-43f7-957d-a3aa5cce4748', 'port_security_enabled': False, 'port_id': 'b5063eaa-2a11-43f7-957d-a3aa5cce4748', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.318 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', 'segmentation_id': 201, 'fixed_ips': [{'subnet_id': 'd7a7901f-8452-4c07-be02-6f4cc1521a4c', 'ip_address': '10.10.10.8'}], 'device_owner': u'network:router_gateway', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:39:39:e4', 'device': u'ac86fc38-efbe-4ca1-86a4-0297e1f358bb', 'port_security_enabled': False, 'port_id': 'ac86fc38-efbe-4ca1-86a4-0297e1f358bb', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.318 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e', 'segmentation_id': 200, 'fixed_ips': [{'subnet_id': '840940c3-f766-4dc2-b2e8-8d3eb0ceb20f', 'ip_address': '192.168.1.2'}], 'device_owner': u'network:dhcp', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:32:40:3d', 'device': u'3aa023f7-f998-4502-b699-dfba9a21045c', 'port_security_enabled': False, 'port_id': '3aa023f7-f998-4502-b699-dfba9a21045c', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.319 10631 DEBUG neutron.agent.rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Returning: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e', 'segmentation_id': 200, 'fixed_ips': [{'subnet_id': '840940c3-f766-4dc2-b2e8-8d3eb0ceb20f', 'ip_address': '192.168.1.7'}], 'device_owner': u'network:router_centralized_snat', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:af:09:ca', 'device': u'4f346d4c-eb4c-42c3-be21-5a86e7cb7097', 'port_security_enabled': False, 'port_id': '4f346d4c-eb4c-42c3-be21-5a86e7cb7097', 'network_type': u'vlan', 'security_groups': []} get_device_details /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/agent/rpc.py:264
2017-10-31 14:20:24.320 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): ListPortsCommand(bridge=br-int) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.321 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.322 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbListCommand(if_exists=True, records=[u'qr-b5063eaa-2a', u'sg-4f346d4c-eb', u'qg-ac86fc38-ef', u'fg-3af2714c-ee', u'tap3aa023f7-f9', u'int-br-provider', u'tapce4f293c-d4'], table=Interface, columns=['name', 'external_ids', 'ofport'], row=False) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.323 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.324 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Processing port: ce4f293c-d486-4973-b9b7-29ad9bee7eee treat_devices_added_or_updated /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1527
2017-10-31 14:20:24.324 10631 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Port ce4f293c-d486-4973-b9b7-29ad9bee7eee updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', 'segmentation_id': 201, 'fixed_ips': [{'subnet_id': 'd7a7901f-8452-4c07-be02-6f4cc1521a4c', 'ip_address': '10.10.10.2'}], 'device_owner': u'network:dhcp', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:46:2e:7d', 'device': u'ce4f293c-d486-4973-b9b7-29ad9bee7eee', 'port_security_enabled': False, 'port_id': 'ce4f293c-d486-4973-b9b7-29ad9bee7eee', 'network_type': u'vlan', 'security_groups': []}
2017-10-31 14:20:24.326 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbGetCommand(column=other_config, table=Port, record=tapce4f293c-d4) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.326 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.327 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbSetCommand(table=Port, col_values=(('other_config', {u'net_uuid': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', u'segmentation_id': '201', u'tag': u'1', u'physical_network': 'physnet1', u'network_type': u'vlan'}),), record=tapce4f293c-d4) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.328 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.329 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Processing port: 3af2714c-eec6-4dcd-b44b-e2fbb08d8eca treat_devices_added_or_updated /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1527
2017-10-31 14:20:24.329 10631 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Port 3af2714c-eec6-4dcd-b44b-e2fbb08d8eca updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', 'segmentation_id': 201, 'fixed_ips': [{'subnet_id': 'd7a7901f-8452-4c07-be02-6f4cc1521a4c', 'ip_address': '10.10.10.6'}], 'device_owner': u'network:floatingip_agent_gateway', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:93:a8:1a', 'device': u'3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', 'port_security_enabled': False, 'port_id': '3af2714c-eec6-4dcd-b44b-e2fbb08d8eca', 'network_type': u'vlan', 'security_groups': []}
2017-10-31 14:20:24.330 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbGetCommand(column=other_config, table=Port, record=fg-3af2714c-ee) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.331 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.332 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbSetCommand(table=Port, col_values=(('other_config', {u'net_uuid': '9ce308e0-7232-4e8d-9f87-d8fe309c7964', u'segmentation_id': '201', u'tag': u'1', u'physical_network': 'physnet1', u'network_type': u'vlan'}),), record=fg-3af2714c-ee) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.333 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.333 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Processing port: b5063eaa-2a11-43f7-957d-a3aa5cce4748 treat_devices_added_or_updated /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py:1527
2017-10-31 14:20:24.334 10631 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] Port b5063eaa-2a11-43f7-957d-a3aa5cce4748 updated. Details: {'profile': {}, 'network_qos_policy_id': None, 'qos_policy_id': None, 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': '93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e', 'segmentation_id': 200, 'fixed_ips': [{'subnet_id': '840940c3-f766-4dc2-b2e8-8d3eb0ceb20f', 'ip_address': '192.168.1.1'}], 'device_owner': u'network:router_interface_distributed', 'physical_network': u'physnet1', 'mac_address': 'fa:16:3e:7f:d4:b5', 'device': u'b5063eaa-2a11-43f7-957d-a3aa5cce4748', 'port_security_enabled': False, 'port_id': 'b5063eaa-2a11-43f7-957d-a3aa5cce4748', 'network_type': u'vlan', 'security_groups': []}
2017-10-31 14:20:24.335 10631 DEBUG neutron.api.rpc.handlers.dvr_rpc [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] neutron.api.rpc.handlers.dvr_rpc.DVRServerRpcApi method get_ports_on_host_by_subnet called with arguments (<neutron_lib.context.ContextBase object at 0x7f309708f790>, 'aio1-neutron-agents-container-ece80330', '840940c3-f766-4dc2-b2e8-8d3eb0ceb20f') {} wrapper /openstack/venvs/neutron-master/lib/python2.7/site-packages/oslo_log/helpers.py:66
2017-10-31 14:20:24.412 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.ovs_dvr_neutron_agent [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] DVR: List of ports received from get_ports_on_host_by_subnet [{u'status': u'ACTIVE', u'name': u'', u'admin_state_up': True, u'network_id': u'93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e', u'tenant_id': u'50a21f82d79a4c018dd84f2c034260dc', u'device_owner': u'network:dhcp', u'mac_address': u'fa:16:3e:32:40:3d', u'project_id': u'50a21f82d79a4c018dd84f2c034260dc', u'fixed_ips': [{u'subnet_id': u'840940c3-f766-4dc2-b2e8-8d3eb0ceb20f', u'ip_address': u'192.168.1.2'}], u'id': u'3aa023f7-f998-4502-b699-dfba9a21045c', u'device_id': u'dhcpb6c0b2d1-de5b-5471-a468-968cca3050a0-93bf45df-63d1-4ef7-b1ab-e67db1ff5b0e'}] _bind_distributed_router_interface_port /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py:387
2017-10-31 14:20:24.415 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): ListPortsCommand(bridge=br-int) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.416 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.419 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn command(idx=0): DbListCommand(if_exists=True, records=[u'qr-b5063eaa-2a', u'sg-4f346d4c-eb', u'qg-ac86fc38-ef', u'fg-3af2714c-ee', u'tap3aa023f7-f9', u'int-br-provider', u'tapce4f293c-d4'], table=Interface, columns=['name', 'external_ids', 'ofport'], row=False) do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2017-10-31 14:20:24.422 10631 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change do_commit /openstack/venvs/neutron-master/lib/python2.7/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:110
2017-10-31 14:20:24.425 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] ofctl request version=0x4,msg_type=0xe,msg_len=0x68,xid=0x4ab14d84,OFPFlowMod(buffer_id=4294967295,command=0,cookie=12551188769968443967L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[OFPInstructionActions(actions=[OFPActionSetField(eth_src=u'fa:16:3e:7f:d4:b5')],len=24,type=4), OFPInstructionGotoTable(len=8,table_id=60,type=1)],match=OFPMatch(oxm_fields={'eth_dst': 'fa:16:3e:32:40:3d', 'vlan_vid': 4296}),out_group=0,out_port=0,priority=4,table_id=2) result None _send_msg /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:99
2017-10-31 14:20:24.427 10631 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-9f88e2a1-069e-4321-865f-e95c97575c45 - - - - -] ofctl request version=0x4,msg_type=0xe,msg_len=0x68,xid=0x4ab14d86,OFPFlowMod(buffer_id=4294967295,command=0,cookie=12551188769968443967L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[OFPInstructionActions(actions=[OFPActionPopVlan(len=8,type=18), OFPActionOutput(len=16,max_len=0,port=4,type=0)],len=32,type=4)],match=OFPMatch(oxm_fields={'eth_dst': 'fa:16:3e:32:40:3d', 'vlan_vid': 4296}),out_group=0,out_port=0,priority=4,table_id=60) result None _send_msg /openstack/venvs/neutron-master/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py:99
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager [-] ofctl_service: Exception occurred during handler processing. Backtrace from offending handler [_handle_send_msg] servicing event [SendMsgRequest] follows.: TypeError: object of type 'NoneType' has no len()
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager Traceback (most recent call last):
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/base/app_manager.py", line 290, in _event_loop
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     handler(ev)
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/app/ofctl/service.py", line 141, in _handle_send_msg
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     datapath.send_msg(msg)
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/controller/controller.py", line 347, in send_msg
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     msg.serialize()
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/ofproto/ofproto_parser.py", line 270, in serialize
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     self._serialize_body()
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/ofproto/ofproto_v1_3_parser.py", line 2708, in _serialize_body
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     match_len = self.match.serialize(self.buf, offset)
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/ofproto/ofproto_v1_3_parser.py", line 1029, in serialize
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     field_offset)
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager   File "/openstack/venvs/neutron-master/lib/python2.7/site-packages/ryu/ofproto/oxx_fields.py", line 253, in _serialize
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager     value_len = len(value)
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager TypeError: object of type 'NoneType' has no len()
2017-10-31 14:20:24.429 10631 ERROR ryu.base.app_manager