contrail-control crashed @ BgpInstanceConfig::service_chain_info

[Jeba Paulaiyan]

contrail-control crashed in Juno Multi node Sanity

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-control'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 BgpInstanceConfig::service_chain_info (this=0x0, family=family@entry=Address::INET) at controller/src/bgp/bgp_config.cc:283
283 controller/src/bgp/bgp_config.cc: No such file or directory.
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
(gdb) bt
#0 BgpInstanceConfig::service_chain_info (this=0x0, family=family@entry=Address::INET) at controller/src/bgp/bgp_config.cc:283
#1 0x000000000060102c in ServiceChainMgr<ServiceChainInet>::ResolvePendingServiceChain (this=0x2016380)
    at controller/src/bgp/routing-instance/service_chaining.cc:1088
#2 0x0000000000bf2a7f in operator() (this=<optimized out>) at /usr/include/boost/function/function_template.hpp:767
#3 TaskTrigger::WorkerTask::Run (this=0x7f3384012b80) at controller/src/base/task_trigger.cc:19
#4 0x0000000000beea3c in TaskImpl::execute (this=0x7f33a2687140) at controller/src/base/task.cc:253
#5 0x00007f33a088eb3a in ?? () from /usr/lib/libtbb.so.2
#6 0x00007f33a088a816 in ?? () from /usr/lib/libtbb.so.2
#7 0x00007f33a0889f4b in ?? () from /usr/lib/libtbb.so.2
#8 0x00007f33a08860ff in ?? () from /usr/lib/libtbb.so.2
#9 0x00007f33a08862f9 in ?? () from /usr/lib/libtbb.so.2
#10 0x00007f33a0aaa182 in start_thread (arg=0x7f3393bfe700) at pthread_create.c:312
#11 0x00007f339fb7b47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)

[Jeba Paulaiyan (jebap)]

core file copied to 10.84.5.112:/cs-shared/bugs/1533438

[Prakash Bailkeri (prakashmb)]

Can't find the core file in the location? Also, please specify the build info.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16284
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16284
Committed: http://github.org/Juniper/contrail-controller/commit/1cde73b8a9db538b70e070c46855cf34f162b28d
Submitter: Zuul
Branch: master

commit 1cde73b8a9db538b70e070c46855cf34f162b28d
Author: Nischal Sheth <email address hidden>
Date: Wed Jan 13 14:22:21 2016 -0800

Fix corner case in routing instance deletion

Problem happens in the following scenario:

- Routing instance has service chain configuration
- Routing instance is on pending queue in service chain manager
- Routing instance is deleted but Shutdown method is not yet called
- Task trigger to resolve pending queue runs

At this time the routing instance has no configuration since it's been
marked deleted but it's still on the pending queue as Shutdown has not
been called. This makes ServiceChainMgr<T>::ResolvePendingServiceChain
dereference the routing instance's NULL config pointer.

This issue was introduced as part of code restructuring related to v6
service chaining. Fix by ensuring that the routing instance is removed
from the pending queue when it's marked deleted.

Credit to Prakash for figuring out the root cause.

Change-Id: I13617622a03360f1df30aa26396801adc704f50b
Closes-Bug: 1533438