mainlinebuild :3037:vrouter agent gets stuck at initialising state and multiple cores at FlowProto::FlowCount

[sundarkh]

mainline3037:vrouter agent core FlowProto::FlowCount

Provisision of a target node with mainline build 3037 leads to vrouter agent core as follows

oot@nodeh8:/var/crashes# gdb ~/contrail-vrouter-agent core.contrail-vroute.9966.nodeh8.1486614064
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /root/contrail-vrouter-agent...done.

warning: core file may not match specified executable file.
[New LWP 9984]
[New LWP 9987]
[New LWP 9985]
[New LWP 9986]
[New LWP 9990]
[New LWP 9988]
[New LWP 9991]
[New LWP 9966]
[New LWP 9989]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 FlowProto::FlowCount (this=0x0) at controller/src/vnsw/agent/pkt/flow_proto.cc:273
273 controller/src/vnsw/agent/pkt/flow_proto.cc: No such file or directory.
(gdb) bt
#0 FlowProto::FlowCount (this=0x0) at controller/src/vnsw/agent/pkt/flow_proto.cc:273
#1 0x0000000000db36a8 in VrouterUveEntry::SendVrouterMsg (this=0x7f40c80043e0) at controller/src/vnsw/agent/uve/vrouter_uve_entry.cc:200
#2 0x0000000000ff1106 in VrouterStatsCollector::Run (this=0x7f40c8002690) at controller/src/vnsw/agent/uve/vrouter_stats_collector.cc:24
#3 0x0000000000cf062a in StatsCollector::TimerExpiry (this=<optimized out>) at controller/src/vnsw/agent/uve/stats_collector.h:83
#4 0x00000000013e5da9 in operator() (this=<optimized out>) at /usr/include/boost/function/function_template.hpp:767
#5 Timer::TimerTask::Run (this=0x309c7c0) at controller/src/base/timer.cc:44
#6 0x00000000013debbf in TaskImpl::execute (this=0x7f40de1bbc40) at controller/src/base/task.cc:276
#7 0x00007f40e5752b3a in ?? () from /usr/lib/libtbb.so.2
#8 0x00007f40e574e816 in ?? () from /usr/lib/libtbb.so.2
#9 0x00007f40e574df4b in ?? () from /usr/lib/libtbb.so.2
#10 0x00007f40e574a0ff in ?? () from /usr/lib/libtbb.so.2
#11 0x00007f40e574a2f9 in ?? () from /usr/lib/libtbb.so.2
#12 0x00007f40e596e184 in start_thread (arg=0x7f40ddfe6700) at pthread_create.c:312
#13 0x00007f40e4c4737d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)

core placed in

10.204.216.49(stack/stack@123)

 /cs-shared/test_runs/nodeh8/jenkins-ServerManager_ubuntu-14-04_mitaka_Single_Node_Sanity-132

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28884
Submitter: jayaramsatya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28884
Committed: http://github.org/Juniper/contrail-sandesh/commit/be1383dd3c69548720140f88c35fe1a6e7cc0f65
Submitter: Zuul (<email address hidden>)
Branch: master

commit be1383dd3c69548720140f88c35fe1a6e7cc0f65
Author: jayaramsatya <email address hidden>
Date: Fri Feb 17 14:35:21 2017 +0530

Added API's directly write/read binary to and from File.
earlier we are calculating size of the buffer approximately and passing
that buffer to Write/Read binary api's some times if there any miss
calculation. buffer over writes happening.
closes-bug: #1663132

Change-Id: I984bff623831eede9965c6ee600f13d88742799c