Juniper Openstack

vcenter-build-19-mitaka-agent crash@FlowProto::FlowEventHandler(FlowEvent*, FlowTable*) ()

Bug #1612049 reported by shajuvk
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
High
Manish Singh
Juniper Openstack r3.0.3.0
R3.1
Fix Committed
High
Manish Singh
Juniper Openstack r3.1.0.0-fcs
Trunk
Fix Committed
High
Manish Singh
Juniper Openstack r3.2.0.0-fcs "r3.2.0.0"

Bug Description

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000000000d056be in ?? ()
(gdb) bt
#0 0x0000000000d056be in ?? ()
#1 0x0000000000d05890 in ?? ()
#2 0x0000000000d07883 in PktFlowInfo::IngressProcess(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#3 0x0000000000d083a5 in PktFlowInfo::Process(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#4 0x0000000000d17863 in FlowHandler::Run() ()
#5 0x0000000000d10b94 in Proto::ProcessProto(boost::shared_ptr<PktInfo>) ()
#6 0x0000000000cf4673 in FlowProto::FlowEventHandler(FlowEvent*, FlowTable*) ()
#7 0x0000000000d14b3d in FlowEventQueueBase::Handler(FlowEvent*) ()
#8 0x0000000000cf93d2 in QueueTaskRunner<FlowEvent*, WorkQueue<FlowEvent*> >::RunQueue() ()
#9 0x00000000012e4dff in TaskImpl::execute() ()
#10 0x00007fd0c5998b3a in ?? () from /usr/lib/libtbb.so.2
#11 0x00007fd0c5994816 in ?? () from /usr/lib/libtbb.so.2
#12 0x00007fd0c5993f4b in ?? () from /usr/lib/libtbb.so.2
#13 0x00007fd0c59900ff in ?? () from /usr/lib/libtbb.so.2
#14 0x00007fd0c59902f9 in ?? () from /usr/lib/libtbb.so.2
#15 0x00007fd0c5bb4184 in start_thread (arg=0x7fd0bd84b700) at pthread_create.c:312
#16 0x00007fd0c4e8d37d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Tags: vrouter
Revision history for this message
shajuvk (shajuvk) wrote :

core are copied to : /auto/cores/shajuvk/1612049

Revision history for this message
shajuvk (shajuvk) wrote :

Build : 3.1.0.0-19
CoreLocation : /cs-shared/test_runs/a6s32/2016_08_10_13_29_52
cores : {'10.84.13.44': ['sh:', '/bin/bash:', 'not', 'found'], '10.84.13.22': ['sh:', '/bin/bash:', 'not', 'found'], '10.84.13.223': ['core.contrail-vroute.1339.ContrailVM-a6s44.1470869997']}
LogsLocation : http://10.204.216.50/Docs/logs/3.1.0.0-19_2016_08_10_13_29_52/logs/
Report : http://10.204.216.50/Docs/logs/3.1.0.0-19_2016_08_10_13_29_52/junit-noframes.html
Topology :
Config Nodes : [u'a6s32', u'a6s33']
Control Nodes : [u'a6s32', u'a6s38']
Compute Nodes : [u'a6s2', u'ContrailVM-a6s22', u'ContrailVM-a6s44']
Openstack Node : [u'a6s32']
WebUI Node : a6s32
Analytics Nodes : [u'a6s32']
Physical Devices : [u"'a5-mx80-2'"]

tags: added: vrouter
summary: - vcenter-build-19-agent crash@FlowProto::FlowEventHandler(FlowEvent*,
- FlowTable*) ()
+ vcenter-build-19-mitaka-agent
+ crash@FlowProto::FlowEventHandler(FlowEvent*, FlowTable*) ()
information type: Proprietary → Public
Revision history for this message
Hari Prasad Killi (haripk) wrote :

(gdb) bt
#0 NhDecode (nh=nh@entry=0x7fd0983a8db0, pkt=pkt@entry=0x7fd09cb94220, info=info@entry=0x7fd0bd84a880, in=in@entry=0x7fd0bd84a800, out=out@entry=0x7fd0bd84a840,
    force_vmport=force_vmport@entry=false, ecmp_load_balance=...) at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:240
#1 0x0000000000d05890 in NhDecode (ecmp_load_balance=..., force_vmport=false, out=0x7fd0bd84a840, in=0x7fd0bd84a800, info=0x7fd0bd84a880, pkt=0x7fd09cb94220, nh=0x7fd0983a8db0)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:415
#2 RouteToOutInfo (rt=<optimized out>, pkt=pkt@entry=0x7fd09cb94220, info=info@entry=0x7fd0bd84a880, in=in@entry=0x7fd0bd84a800, out=out@entry=0x7fd0bd84a840)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:415
#3 0x0000000000d07883 in PktFlowInfo::IngressProcess (this=this@entry=0x7fd0bd84a880, pkt=pkt@entry=0x7fd09cb94220, in=in@entry=0x7fd0bd84a800, out=out@entry=0x7fd0bd84a840)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:1167
#4 0x0000000000d083a5 in PktFlowInfo::Process (this=this@entry=0x7fd0bd84a880, pkt=0x7fd09cb94220, in=in@entry=0x7fd0bd84a800, out=out@entry=0x7fd0bd84a840)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:1466
#5 0x0000000000d17863 in FlowHandler::Run (this=0x7fd09c9cb820) at controller/src/vnsw/agent/pkt/flow_handler.cc:114
#6 0x0000000000d10b94 in RunProtoHandler (handler=0x7fd09c9cb820, this=0x7fd0a8af9370) at controller/src/vnsw/agent/pkt/proto.cc:51
#7 Proto::ProcessProto (this=this@entry=0x7fd0a8af9370, msg_info=(boost::shared_ptr<PktInfo>) (count 5, weak count 1) 0x7fd09cb94220) at controller/src/vnsw/agent/pkt/proto.cc:66
#8 0x0000000000cf4673 in FlowProto::FlowEventHandler (this=0x7fd0a8af9370, req=0x7fd09caa80e0, table=<optimized out>) at controller/src/vnsw/agent/pkt/flow_proto.cc:406
#9 0x0000000000d14b3d in FlowEventQueueBase::Handler (this=0x7fd09c8019f0, event=0x7fd09caa80e0) at controller/src/vnsw/agent/pkt/flow_event.cc:105
#10 0x0000000000cf93d2 in operator() (a0=0x7fd09caa80e0, this=0x7fd0bd84ab30) at /usr/include/boost/function/function_template.hpp:767
#11 QueueTaskRunner<FlowEvent*, WorkQueue<FlowEvent*> >::RunQueue (this=0x7fd09ca81090) at controller/src/base/queue_task.h:92

(gdb) p out->intf_->flow_key_nh_
$3 = (boost::intrusive_ptr<NextHop const>) 0x0

(gdb) p $2->layer3_forwarding_
$5 = true

(gdb) p $2->ipv4_active_
$6 = false

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/23204
Submitter: Manish Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/23204
Committed: http://github.org/Juniper/contrail-controller/commit/8e4e442ee20c9e6acaef635214a7464c28fa25d4
Submitter: Zuul
Branch: R3.1

commit 8e4e442ee20c9e6acaef635214a7464c28fa25d4
Author: Manish <email address hidden>
Date: Thu Aug 11 15:02:50 2016 +0530

Agent crash@FlowProto::FlowEventHandler

Problem:
Interface was transitioning its oper ip forwarding state. Flow was being created
using same interface, however because of transition interface didnt add l3 nh to
flow_key of vmi. Flow requested for flow key which was NULL and crashed.

Solution:Create L3 nh based on l3 config in vmi.

Change-Id: I0d4a972ffb9238c6459729d0171c3d6b0e343a20
Closes-bug: #1612049

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/23360
Submitter: Manish Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/23402
Submitter: Manish Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/23360
Committed: http://github.org/Juniper/contrail-controller/commit/3b44ff213ef962437b06dd5507421e335e889c21
Submitter: Zuul
Branch: master

commit 3b44ff213ef962437b06dd5507421e335e889c21
Author: Manish <email address hidden>
Date: Thu Aug 11 15:02:50 2016 +0530

Agent crash@FlowProto::FlowEventHandler

Problem:
Interface was transitioning its oper ip forwarding state. Flow was being created
using same interface, however because of transition interface didnt add l3 nh to
flow_key of vmi. Flow requested for flow key which was NULL and crashed.

Solution:Create L3 nh based on l3 config in vmi.

Change-Id: I0d4a972ffb9238c6459729d0171c3d6b0e343a20
Closes-bug: #1612049
(cherry picked from commit 8e4e442ee20c9e6acaef635214a7464c28fa25d4)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/23402
Committed: http://github.org/Juniper/contrail-controller/commit/f73a2d0ca4b97522f4e9c2b065b0b700031bf5b4
Submitter: Zuul
Branch: R3.0

commit f73a2d0ca4b97522f4e9c2b065b0b700031bf5b4
Author: Manish <email address hidden>
Date: Thu Aug 11 15:02:50 2016 +0530

Agent crash@FlowProto::FlowEventHandler

Problem:
Interface was transitioning its oper ip forwarding state. Flow was being created
using same interface, however because of transition interface didnt add l3 nh to
flow_key of vmi. Flow requested for flow key which was NULL and crashed.

Solution:Create L3 nh based on l3 config in vmi.

Closes-bug: #1612049
(cherry picked from commit 8e4e442ee20c9e6acaef635214a7464c28fa25d4)

Conflicts:
 src/vnsw/agent/oper/vm_interface.cc
Change-Id: I0d4a972ffb9238c6459729d0171c3d6b0e343a20

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.