2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_log/helpers.py", line 67, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return method(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py", line 41, in add_fdb_entries 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server self.fdb_add(context, self._unmarshall_fdb_entries(fdb_entries)) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 524, in fdb_add 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server agent_ports, self._tunnel_port_lookup) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_log/helpers.py", line 67, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return method(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py", line 254, in fdb_add_tun 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server self.add_fdb_flow(br, port, remote_ip, lvm, ofport) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 153, in wrapper 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 552, in add_fdb_flow 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server port_info.mac_address) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_tun.py", line 203, in install_unicast_to_tun 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server actions=actions) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 227, in install_apply_actions 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server **match_kwargs) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 214, in install_instructions 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server self._send_msg(msg) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 95, in _send_msg 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server raise RuntimeError(m) 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server RuntimeError: ofctl request version=0x4,msg_type=0xe,msg_len=0x78,xid=0x886c53aa,OFPFlowMod(buffer_id=4294967295,command=0,cookie=13128218601586226859L,cookie_mask=0,flags=0,hard_timeout=0,idle_timeout=0,instructions=[OFPInstructionActions(actions=[OFPActionPopVlan(len=8,type=18), OFPActionSetField(tunnel_id=155), OFPActionOutput(len=16,max_len=0,port=5,type=0)],len=48,type=4)],match=OFPMatch(oxm_fields={'eth_dst': 'fa:16:3e:af:d2:95', 'vlan_vid': 4368}),out_group=0,out_port=0,priority=2,table_id=20) timed out 2018-05-30 20:05:32.387 7 ERROR oslo_messaging.rpc.server
2018-05-30 20:05:32.387 7 ERROR oslo_messaging. rpc.server Traceback (most recent call last): rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/server. py", line 160, in _process_incoming rpc.server res = self.dispatcher .dispatch( message) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 213, in dispatch rpc.server return self._do_ dispatch( endpoint, method, ctxt, args) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_messaging/ rpc/dispatcher. py", line 183, in _do_dispatch rpc.server result = func(ctxt, **new_args) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ osprofiler/ profiler. py", line 153, in wrapper rpc.server return f(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_log/ helpers. py", line 67, in wrapper rpc.server return method(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ l2pop/rpc_ manager/ l2population_ rpc.py" , line 41, in add_fdb_entries rpc.server self.fdb_ add(context, self._unmarshal l_fdb_entries( fdb_entries) ) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ osprofiler/ profiler. py", line 153, in wrapper rpc.server return f(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/ovs_ neutron_ agent.py" , line 524, in fdb_add rpc.server agent_ports, self._tunnel_ port_lookup) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ osprofiler/ profiler. py", line 153, in wrapper rpc.server return f(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_log/ helpers. py", line 67, in wrapper rpc.server return method(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ l2pop/rpc_ manager/ l2population_ rpc.py" , line 254, in fdb_add_tun rpc.server self.add_ fdb_flow( br, port, remote_ip, lvm, ofport) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ osprofiler/ profiler. py", line 153, in wrapper rpc.server return f(*args, **kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/ovs_ neutron_ agent.py" , line 552, in add_fdb_flow rpc.server port_info. mac_address) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/openflow/ native/ br_tun. py", line 203, in install_ unicast_ to_tun rpc.server actions=actions) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/openflow/ native/ ofswitch. py", line 227, in install_ apply_actions rpc.server **match_kwargs) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/openflow/ native/ ofswitch. py", line 214, in install_ instructions rpc.server self._send_msg(msg) rpc.server File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ neutron/ plugins/ ml2/drivers/ openvswitch/ agent/openflow/ native/ ofswitch. py", line 95, in _send_msg rpc.server raise RuntimeError(m) rpc.server RuntimeError: ofctl request version= 0x4,msg_ type=0xe, msg_len= 0x78,xid= 0x886c53aa, OFPFlowMod( buffer_ id=4294967295, command= 0,cookie= 131282186015862 26859L, cookie_ mask=0, flags=0, hard_timeout= 0,idle_ timeout= 0,instructions= [OFPInstruction Actions( actions= [OFPActionPopVl an(len= 8,type= 18), OFPActionSetFie ld(tunnel_ id=155) , OFPActionOutput (len=16, max_len= 0,port= 5,type= 0)],len= 48,type= 4)],match= OFPMatch( oxm_fields= {'eth_dst' : 'fa:16: 3e:af:d2: 95', 'vlan_vid': 4368}), out_group= 0,out_port= 0,priority= 2,table_ id=20) timed out rpc.server
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.
2018-05-30 20:05:32.387 7 ERROR oslo_messaging.