Agent crash in virtual tbb::task* TaskImpl::execute()

[amit surana]

core file is in 10.84.5.112:/cs-shared/bugs/<bug-id>/

3.0.2.0-26~kilo

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007fe803c99cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
(gdb) bt
#0 0x00007fe803c99cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007fe803c9d0d8 in __GI_abort () at abort.c:89
#2 0x00007fe803c92b86 in __assert_fail_base (fmt=0x7fe803de3830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x11d66d5 "0", file=file@entry=0x12cd4e0 "controller/src/base/task.cc", line=line@entry=276,
    function=function@entry=0x12cec00 "virtual tbb::task* TaskImpl::execute()") at assert.c:92
#3 0x00007fe803c92c32 in __GI___assert_fail (assertion=0x11d66d5 "0", file=0x12cd4e0 "controller/src/base/task.cc", line=276,
    function=0x12cec00 "virtual tbb::task* TaskImpl::execute()") at assert.c:101
#4 0x0000000001186d33 in TaskImpl::execute() ()
#5 0x00007fe804868b3a in ?? () from /usr/lib/libtbb.so.2
#6 0x00007fe804864816 in ?? () from /usr/lib/libtbb.so.2
#7 0x00007fe804863f4b in ?? () from /usr/lib/libtbb.so.2
#8 0x00007fe8048600ff in ?? () from /usr/lib/libtbb.so.2
#9 0x00007fe8048602f9 in ?? () from /usr/lib/libtbb.so.2
#10 0x00007fe804a84182 in start_thread (arg=0x7fe7cb2fc700) at pthread_create.c:312
#11 0x00007fe803d5d47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) q

[Hari Prasad Killi (haripk)]

"std::bad_alloc" exception is causing this crash - memory allocation failure causes this exception. The core file size was only about 1.3 G. Not sure if we can conclude that there is a memory leak based on this.

What was being tested when this core was seen. Is this repeatable. We need to fetch the memory usage data of the vrouter-agent from analytics.

[amit surana (asurana-t)]

This was seen once over a 2 day run of solution tests. Will get the additional data if this is seen again.

[amit surana (asurana-t)]

crash was seen again while creating/deleting service-chains. core placed in the same location.

[Hari Prasad Killi (haripk)]

Please check if the issue is seen after the stability fixes went in, expecting this exception to be due to corruption caused by earlier bug.

[Hari Prasad Killi (haripk)]

Marking as fixed, as the issue is not seen after the stability fixes.