3.1 build 2 : vroute agent core over a config node with FlowMgmtDbClient::RouteNotify(FlowMgmtDbClient::VrfFlowHandlerState*, Agent::RouteTableType, DBTablePartBase*, DBEntryBase*)

[sundarkh]

3.1 build 2 : vroute agent core over a config node

Topology : Multi NOde HA

Config Nodes : [u'nodeg21', u'nodec58']
Control Nodes : [u'nodeg33', u'nodec58']
Compute Nodes : [u'nodec38', u'nodeg17']
Openstack Node : nodeg33
WebUI Node : nodeg21
Analytics Nodes : [u'nodeg21']
Physical Devices : [u"'mx2'"]

during Sanity run, got generated
http://anamika.englab.juniper.net:8080/job/ServerManager_ubuntu-14-04_kilo_multi_Node_Sanity/180/consoleFull

Backtrace :
-----------
root@nodeg17:/var/crashes# gdb contrail-vrouter-agent nodeg17_core.contrail-vroute.4363.nodeg17.1468516020
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from contrail-vrouter-agent...(no debugging symbols found)...done.

warning: core file may not match specified executable file.
[New LWP 4381]
[New LWP 4384]
[New LWP 4363]
[New LWP 4382]
[New LWP 7343]
[New LWP 4380]
[New LWP 4383]
[New LWP 4385]
[New LWP 4387]
[New LWP 4386]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000000000ccfe77 in FlowMgmtDbClient::RouteNotify(FlowMgmtDbClient::VrfFlowHandlerState*, Agent::RouteTableType, DBTablePartBase*, DBEntryBase*) ()
(gdb) bt
#0 0x0000000000ccfe77 in FlowMgmtDbClient::RouteNotify(FlowMgmtDbClient::VrfFlowHandlerState*, Agent::RouteTableType, DBTablePartBase*, DBEntryBase*) ()
#1 0x000000000115881a in DBTableBase::RunNotify(DBTablePartBase*, DBEntryBase*) ()
#2 0x000000000115afd8 in DBTablePartBase::RunNotify() ()
#3 0x0000000001156c3e in DBPartition::QueueRunner::Run() ()
#4 0x000000000128228f in TaskImpl::execute() ()
#5 0x00007f1bc8c91b3a in ?? () from /usr/lib/libtbb.so.2
#6 0x00007f1bc8c8d816 in ?? () from /usr/lib/libtbb.so.2
#7 0x00007f1bc8c8cf4b in ?? () from /usr/lib/libtbb.so.2
#8 0x00007f1bc8c890ff in ?? () from /usr/lib/libtbb.so.2
#9 0x00007f1bc8c892f9 in ?? () from /usr/lib/libtbb.so.2
#10 0x00007f1bc8ead182 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#11 0x00007f1bc818647d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) n

it is avaialble in anamika server
----------------------------------
cs-shared/test_runs/nodeg21/2016_07_14_21_14_13

[sundarkh (sundar-kh)]

Bug got created with test_ecmp_svc_in_network_with_static_route_no_policy Test case

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22330
Submitter: Manish Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22332
Submitter: Manish Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22332
Committed: http://github.org/Juniper/contrail-controller/commit/df2ebb63b349277da4a4aa6437432a1baa0d8392
Submitter: Zuul
Branch: R3.1

commit df2ebb63b349277da4a4aa6437432a1baa0d8392
Author: Manish <email address hidden>
Date: Fri Jul 22 11:46:26 2016 +0530

Agent crashed at FlowMgmtDbClient::RouteNotify

Problem:
Subnet route deletion enqueues request to resync all host route which were
pointing to it(because of inet component of evpn route). This enqueue resets the
dependant_rt under host route. However before this request is processed, call
happens for getting active path on host route. Since dependant_rt is not yet
reset, call tries to use active path of same which in turns return NULL as route
is marked for deletion. This NULL causes crash.

Solution:
Return self if path is NULL from dependant_rt.

Change-Id: Ic108cad386b74105a0084726f184debc662c2675
Closes-bug: #1603388

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22330
Committed: http://github.org/Juniper/contrail-controller/commit/5a794121b2504bc961a9ca94fff762affcf89557
Submitter: Zuul
Branch: master

commit 5a794121b2504bc961a9ca94fff762affcf89557
Author: Manish <email address hidden>
Date: Fri Jul 22 11:46:26 2016 +0530

Agent crashed at FlowMgmtDbClient::RouteNotify

Problem:
Subnet route deletion enqueues request to resync all host route which were
pointing to it(because of inet component of evpn route). This enqueue resets the
dependant_rt under host route. However before this request is processed, call
happens for getting active path on host route. Since dependant_rt is not yet
reset, call tries to use active path of same which in turns return NULL as route
is marked for deletion. This NULL causes crash.

Solution:
Return self if path is NULL from dependant_rt.

Change-Id: Ic108cad386b74105a0084726f184debc662c2675
Closes-bug: #1603388