DPDK: agent crash in PktFlowInfo::IngressProcess

[Vinod Nair]

Agent crashed in PktFlowInfo::IngressProcess . Backtrace is as below

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000000000c383ad in ?? ()
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
(gdb) bt
#0 0x0000000000c383ad in ?? ()
#1 0x0000000000c387b0 in ?? ()
#2 0x0000000000c3c354 in PktFlowInfo::IngressProcess(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#3 0x0000000000c3ce4e in PktFlowInfo::Process(PktInfo const*, PktControlInfo*, PktControlInfo*) ()
#4 0x0000000000c48ffd in FlowHandler::Run() ()
#5 0x0000000000c452d4 in Proto::ProcessProto(boost::shared_ptr<PktInfo>) ()
#6 0x0000000000c29e90 in FlowProto::FlowEventHandler(FlowEvent*, FlowTable*) ()
#7 0x0000000000c305af in QueueTaskRunner<FlowEvent*, WorkQueue<FlowEvent*> >::Run() ()
#8 0x000000000118b89c in TaskImpl::execute() ()
#9 0x00007fc2264e9b3a in ?? () from /usr/lib/libtbb.so.2
#10 0x00007fc2264e5816 in ?? () from /usr/lib/libtbb.so.2
#11 0x00007fc2264e4f4b in ?? () from /usr/lib/libtbb.so.2
#12 0x00007fc2264e10ff in ?? () from /usr/lib/libtbb.so.2
#13 0x00007fc2264e12f9 in ?? () from /usr/lib/libtbb.so.2
#14 0x00007fc226705182 in start_thread (arg=0x7fc21db9c700) at pthread_create.c:312
#15 0x00007fc2259de47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)

Build: 3.0.2.0-30~kilo
core: /cs-shared/bugs/1572418

[Hari Prasad Killi (haripk)]

(gdb) bt
#0 NhDecode (nh=<optimized out>, nh@entry=0x7fc2080aafe0, pkt=pkt@entry=0x7fc1708b0110, info=info@entry=0x7fc21db9b8a0, in=in@entry=0x7fc21db9b820, out=out@entry=0x7fc21db9b860,
    force_vmport=force_vmport@entry=false, ecmp_load_balance=...) at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:305
#1 0x0000000000c387b0 in NhDecode (ecmp_load_balance=..., force_vmport=false, out=0x7fc21db9b860, in=0x7fc21db9b820, info=0x7fc21db9b8a0, pkt=0x7fc1708b0110, nh=0x7fc2080aafe0)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:409
#2 RouteToOutInfo (rt=<optimized out>, pkt=pkt@entry=0x7fc1708b0110, info=info@entry=0x7fc21db9b8a0, in=in@entry=0x7fc21db9b820, out=out@entry=0x7fc21db9b860)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:409
#3 0x0000000000c3c354 in PktFlowInfo::IngressProcess (this=this@entry=0x7fc21db9b8a0, pkt=pkt@entry=0x7fc1708b0110, in=in@entry=0x7fc21db9b820, out=out@entry=0x7fc21db9b860)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:1141
#4 0x0000000000c3ce4e in PktFlowInfo::Process (this=this@entry=0x7fc21db9b8a0, pkt=0x7fc1708b0110, in=in@entry=0x7fc21db9b820, out=out@entry=0x7fc21db9b860)
    at controller/src/vnsw/agent/pkt/pkt_flow_info.cc:1424
#5 0x0000000000c48ffd in FlowHandler::Run (this=0x7fc1e32a8820) at controller/src/vnsw/agent/pkt/flow_handler.cc:112
#6 0x0000000000c452d4 in RunProtoHandler (handler=0x7fc1e32a8820, this=0x7fc21db9ba90) at controller/src/vnsw/agent/pkt/proto.cc:51
#7 Proto::ProcessProto (this=this@entry=0x7fc20d027030, msg_info=...) at controller/src/vnsw/agent/pkt/proto.cc:66
#8 0x0000000000c29e90 in FlowProto::FlowEventHandler (this=0x7fc20d027030, req=0x7fc1864e5930, table=<optimized out>) at controller/src/vnsw/agent/pkt/flow_proto.cc:396

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/19734
Submitter: jayaramsatya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/19926
Submitter: jayaramsatya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/19926
Committed: http://github.org/Juniper/contrail-controller/commit/80d6ea3cc1c53503ca7ef94a0a4d23851e4d4415
Submitter: Zuul
Branch: R3.0

commit 80d6ea3cc1c53503ca7ef94a0a4d23851e4d4415
Author: jayaramsatya <email address hidden>
Date: Thu May 5 15:31:37 2016 +0530

Issue:
-----
This Crash is seen becuase while getting the Policy enabled NH, the
interface marked as delete.
Fix:
----
Before getting the Policy enabled NH Check for Interface
active if it is not active, mark that flow as short flow.
closes-bug: #1572418

Change-Id: I493e61e97a1d833af096421ea2c3c3a5a68e66f6

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20914
Submitter: jayaramsatya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20914
Committed: http://github.org/Juniper/contrail-controller/commit/1c94e5441fc17ee243197cf9246822ad9ceea285
Submitter: Zuul
Branch: master

commit 1c94e5441fc17ee243197cf9246822ad9ceea285
Author: jayaramsatya <email address hidden>
Date: Thu May 5 15:31:37 2016 +0530

Issue:
-----
This Crash is seen becuase while getting the Policy enabled NH, the
interface marked as delete.
Fix:
----
Before getting the Policy enabled NH Check for Interface
active if it is not active, mark that flow as short flow.
closes-bug: #1572418

Change-Id: I4b8d0ece581a2963860203777a53c151b2f4dfa0