ofctl request Datapath Invalid errors

Bug #1672610 reported by IWAMOTO Toshihiro
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
neutron
Won't Fix
Undecided
IWAMOTO Toshihiro

Bug Description

openflow connection is sometimes lost and the subsequent openflow command fails. ~70 occurences in a week in logstash search.
Retrying should save such cases.

example:

http://logs.openstack.org/98/436798/25/check/gate-tempest-dsvm-neutron-full-centos-7-nv/3d5e54b/logs/screen-neutron-agent.txt.gz#_2017-03-13_06_54_30_891

2017-03-13 06:54:30.896 16978 ERROR OfctlService [-] unknown dpid 143366513125697
2017-03-13 06:54:30.897 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-a1036712-2598-48a8-8a38-8e7eacc0ed8f None None] ofctl request version=None,msg_type=None,msg_len=None,xid=None,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) error Datapath Invalid 143366513125697
2017-03-13 06:54:30.898 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int [req-a1036712-2598-48a8-8a38-8e7eacc0ed8f None None] Failed to communicate with the switch
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int Traceback (most recent call last):
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_int.py", line 52, in check_canary_table
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int flows = self.dump_flows(constants.CANARY_TABLE)
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 131, in dump_flows
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int reply_multi=True)
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/ofswitch.py", line 79, in _send_msg
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int raise RuntimeError(m)
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int RuntimeError: ofctl request version=None,msg_type=None,msg_len=None,xid=None,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) error Datapath Invalid 143366513125697
2017-03-13 06:54:30.898 16978 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.br_int
2017-03-13 06:54:30.907 WARNING neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-a1036712-2598-48a8-8a38-8e7eacc0ed8f None None] OVS is dead. OVSNeutronAgent will keep running and checking OVS status periodically.

Tags: ovs
Revision history for this message
IWAMOTO Toshihiro (iwamoto) wrote :
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/445308

Changed in neutron:
assignee: nobody → IWAMOTO Toshihiro (iwamoto)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/445308
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Bug closed due to lack of activity, please feel free to reopen if needed.

The other related bugs are fixed and released.

Changed in neutron:
status: In Progress → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.