[2.21-95] Agent crash @ bool VrfEntry::DeleteTimeout()

[chhandak]

Observed the crash in only compute node (no TOR Agent or TSN)
Observed while clearing the setup used for inter VN communication.

BT
---
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-vrouter-agent'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f6409443cc9 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#0 0x00007f6409443cc9 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f64094470d8 in __GI_abort () at abort.c:89
#2 0x00007f640943cb86 in __assert_fail_base (
    fmt=0x7f640958d830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x1012c95 "0",
    file=file@entry=0x1021040 "controller/src/vnsw/agent/oper/vrf.cc",
    line=line@entry=333,
    function=function@entry=0x1021200 "bool VrfEntry::DeleteTimeout()")
    at assert.c:92
#3 0x00007f640943cc32 in __GI___assert_fail (assertion=0x1012c95 "0",
    file=0x1021040 "controller/src/vnsw/agent/oper/vrf.cc", line=333,
    function=0x1021200 "bool VrfEntry::DeleteTimeout()") at assert.c:101
#4 0x00000000009df41d in VrfEntry::DeleteTimeout() ()
#5 0x0000000000fdcbe9 in Timer::TimerTask::Run() ()
#6 0x0000000000fd65e0 in TaskImpl::execute() ()
#7 0x00007f640a012b3a in ?? () from /usr/lib/libtbb.so.2
#8 0x00007f640a00e816 in ?? () from /usr/lib/libtbb.so.2
#9 0x00007f640a00df4b in ?? () from /usr/lib/libtbb.so.2
#10 0x00007f640a00a0ff in ?? () from /usr/lib/libtbb.so.2
#11 0x00007f640a00a2f9 in ?? () from /usr/lib/libtbb.so.2
#12 0x00007f640a22e182 in start_thread (arg=0x7f64026c9700)
    at pthread_create.c:312
#13 0x00007f640950747d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

[chhandak (chhandak)]

Logs saved at http://mayamruga.englab.juniper.net/bugs/1493861

[Daisuke Nakajima (dnakajima)]

same core is seen on ToR-agent
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/contrail-tor-agent --config_file /etc/contrail/contrail-tor-agent-0.co'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007fe412181cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007fe412181cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007fe4121850d8 in __GI_abort () at abort.c:89
#2 0x00007fe41217ab86 in __assert_fail_base (fmt=0x7fe4122cb830 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0xe4c275 "0", file=file@entry=0xe5a640 "controller/src/vnsw/agent/oper/vrf.cc", line=line@entry=333,
    function=function@entry=0xe5a800 "bool VrfEntry::DeleteTimeout()") at assert.c:92
#3 0x00007fe41217ac32 in __GI___assert_fail (assertion=0xe4c275 "0", file=0xe5a640 "controller/src/vnsw/agent/oper/vrf.cc",
    line=333, function=0xe5a800 "bool VrfEntry::DeleteTimeout()") at assert.c:101
#4 0x000000000091b56d in VrfEntry::DeleteTimeout() ()
#5 0x0000000000e1c3f9 in Timer::TimerTask::Run() ()
#6 0x0000000000e15df0 in TaskImpl::execute() ()
#7 0x00007fe412d50b3a in ?? () from /usr/lib/libtbb.so.2
#8 0x00007fe412d4c816 in ?? () from /usr/lib/libtbb.so.2
#9 0x00007fe412d4bf4b in ?? () from /usr/lib/libtbb.so.2
#10 0x00007fe412d480ff in ?? () from /usr/lib/libtbb.so.2
#11 0x00007fe412d482f9 in ?? () from /usr/lib/libtbb.so.2
#12 0x00007fe412f6c182 in start_thread (arg=0x7fe40b808700) at pthread_create.c:312
#13 0x00007fe41224547d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

[OpenContrail Admin (ci-admin-f)]

bug update...

[Manish Singh (manishs)]

Not seen again.