agent crash at pthread_mutex_lock

[Vedamurthy Joshi]

R3.0.2.0 Build 33 with private agent binary from Praveen(for flow setup rate)

Not sure of the trigger, but multiple similar cores were seen on two computes

Core will be in http://10.204.216.50/Docs/bugs/#

(gdb) bt
#0 0x00007fb9e2a73414 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000000000c688c1 in lock (this=0x3a8) at /usr/include/tbb/mutex.h:164
#2 acquire (mutex=..., this=<synthetic pointer>) at /usr/include/tbb/mutex.h:105
#3 scoped_lock (mutex=..., this=<synthetic pointer>) at /usr/include/tbb/mutex.h:91
#4 FlowHandler::Run (this=0x7fb9b4058260) at controller/src/vnsw/agent/pkt/flow_handler.cc:79
#5 0x0000000000c61c6d in Proto::RunProtoHandler (this=this@entry=0x7fb9c8acf550, handler=handler@entry=0x7fb9b4058260) at controller/src/vnsw/agent/pkt/proto.cc:51
#6 0x0000000000c45cd6 in FlowProto::FlowEventHandler (this=0x7fb9c8acf550, req=<optimized out>, table=<optimized out>) at controller/src/vnsw/agent/pkt/flow_proto.cc:393
#7 0x0000000000c65a3d in FlowEventQueueBase::Handler (this=0x7fb9b8407250, event=0x7fb9c4094520) at controller/src/vnsw/agent/pkt/flow_event.cc:103
#8 0x0000000000c4abfa in operator() (a0=0x7fb9c4094520, this=0x7fb9d9705b30) at /usr/include/boost/function/function_template.hpp:767
#9 RunQueue (this=0x7fb9c40945c0) at controller/src/base/queue_task.h:87
#10 QueueTaskRunner<FlowEvent*, WorkQueue<FlowEvent*> >::Run (this=0x7fb9c40945c0) at controller/src/base/queue_task.h:66
#11 0x00000000011b3d8f in TaskImpl::execute (this=0x7fb9db4cfc40) at controller/src/base/task.cc:261
#12 0x00007fb9e2855b3a in ?? () from /usr/lib/libtbb.so.2
#13 0x00007fb9e2851816 in ?? () from /usr/lib/libtbb.so.2
#14 0x00007fb9e2850f4b in ?? () from /usr/lib/libtbb.so.2
#15 0x00007fb9e284d0ff in ?? () from /usr/lib/libtbb.so.2
#16 0x00007fb9e284d2f9 in ?? () from /usr/lib/libtbb.so.2
#17 0x00007fb9e2a71182 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#18 0x00007fb9e1d4a47d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20292
Submitter: Prabhjot Singh Sethi (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20293
Submitter: Prabhjot Singh Sethi (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20292
Committed: http://github.org/Juniper/contrail-controller/commit/8e0a955dad9b8eabc4abe21f88129e5c29f4333d
Submitter: Zuul
Branch: R3.0

commit 8e0a955dad9b8eabc4abe21f88129e5c29f4333d
Author: Prabhjot Singh Sethi <email address hidden>
Date: Tue May 17 14:25:17 2016 +0530

Fix sanity issue

Adding missing break for a switch case which was causing
sanity failures

Closes-Bug: 1582526
Change-Id: I8bf4e56c1c8d4421627209271e0849b969b4974a

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20293
Committed: http://github.org/Juniper/contrail-controller/commit/9d92987d1c66b16789e61b0de12196bad326bf50
Submitter: Zuul
Branch: master

commit 9d92987d1c66b16789e61b0de12196bad326bf50
Author: Prabhjot Singh Sethi <email address hidden>
Date: Tue May 17 14:25:17 2016 +0530

Fix sanity issue

Adding missing break for a switch case which was causing
sanity failures

Closes-Bug: 1582526
Change-Id: I8bf4e56c1c8d4421627209271e0849b969b4974a
(cherry picked from commit 8e0a955dad9b8eabc4abe21f88129e5c29f4333d)