Working on something right now.
As requested, can you please upload any core you see, add that info to the bug along with the version number.
Thanks.

On Jun 23, 2015, at 5:02 AM, Chhandak Mukherjee wrote:

> Hi Tapan.
> 
> This crash can be consistently reproducible .
> 
> When ever we delete logical interface in chunk (using webui), I have hit
> the crash. 
> 
> You can have a look at the crash @ nodei6 (10.204.217.118) root/c0ntrail123
> 
> Thanks and Regards,
> Chhandak 
> 
> 
> 
> On 6/15/15, 10:32 PM, "OpenContrail Admin" <email address hidden>
> wrote:
> 
>> ** Changed in: juniperopenstack/r2.20
>>   Milestone: r2.20-fcs => r2.21
>> 
>> ** Tags added: quench
>> 
>> -- 
>> You received this bug notification because you are a member of Contrail
>> Systems engineering, which is subscribed to Juniper Openstack.
>> https://bugs.launchpad.net/bugs/1453369
>> 
>> Title:
>> [Build R2.20 10 Juno] TOR Scale:Control node crash @
>> IFMapExporter::StateUpdateOnDequeue
>> 
>> Status in Juniper Openstack distribution:
>> New
>> Status in Juniper Openstack r2.20 series:
>> New
>> Status in Juniper Openstack trunk series:
>> New
>> 
>> Bug description:
>> 
>> Trigger
>> -------------
>> System had around 16K Vmi. Deleting all of them in one go with UI.
>> 
>> Backtrace
>> ----------------
>> (gdb) bt
>> #0  0x00007f547dd66cc9 in __GI_raise (sig=sig@entry=6) at
>> ../nptl/sysdeps/unix/sysv/linux/raise.c:56
>> #1  0x00007f547dd6a0d8 in __GI_abort () at abort.c:89
>> #2  0x00007f547dd5fb86 in __assert_fail_base (fmt=0x7f547deb0830
>> "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
>> assertion=assertion@entry=0xadebe1 "state->advertised().empty()",
>>     file=file@entry=0xadec78 "controller/src/ifmap/ifmap_exporter.cc",
>> line=line@entry=548,
>>     function=function@entry=0xadf000
>> <IFMapExporter::StateUpdateOnDequeue(IFMapUpdate*, BitSet const&,
>> bool)::__PRETTY_FUNCTION__> "void
>> IFMapExporter::StateUpdateOnDequeue(IFMapUpdate*, const BitSet&, bool)")
>> at assert.c:92
>> #3  0x00007f547dd5fc32 in __GI___assert_fail (assertion=0xadebe1
>> "state->advertised().empty()", file=0xadec78
>> "controller/src/ifmap/ifmap_exporter.cc", line=548,
>>     function=0xadf000
>> <IFMapExporter::StateUpdateOnDequeue(IFMapUpdate*, BitSet const&,
>> bool)::__PRETTY_FUNCTION__> "void
>> IFMapExporter::StateUpdateOnDequeue(IFMapUpdate*, const BitSet&, bool)")
>> at assert.c:101
>> #4  0x000000000045df76 in IFMapExporter::StateUpdateOnDequeue
>> (this=0x2d92150, update=update@entry=0x7f545688de70, dequeue_set=...,
>> is_delete=<optimized out>)
>>     at controller/src/ifmap/ifmap_exporter.cc:548
>> #5  0x000000000048add2 in IFMapUpdateSender::ProcessUpdate
>> (this=this@entry=0x2d93600, update=update@entry=0x7f545688de70,
>> base_send_set=...)
>>     at controller/src/ifmap/ifmap_update_sender.cc:225
>> #6  0x000000000048b314 in IFMapUpdateSender::Send (this=0x2d93600,
>> imarker=<optimized out>) at
>> controller/src/ifmap/ifmap_update_sender.cc:184
>> #7  0x000000000048badb in IFMapUpdateSender::SendTask::Run
>> (this=0x2dcf520) at controller/src/ifmap/ifmap_update_sender.cc:41
>> #8  0x0000000000ab2ad0 in TaskImpl::execute (this=0x7f54774c2040) at
>> controller/src/base/task.cc:232
>> #9  0x00007f547eb3db3a in ?? () from /usr/lib/libtbb.so.2
>> #10 0x00007f547eb39816 in ?? () from /usr/lib/libtbb.so.2
>> #11 0x00007f547eb38f4b in ?? () from /usr/lib/libtbb.so.2
>> #12 0x00007f547eb350ff in ?? () from /usr/lib/libtbb.so.2
>> #13 0x00007f547eb352f9 in ?? () from /usr/lib/libtbb.so.2
>> #14 0x00007f547ed59182 in start_thread (arg=0x7f544dbf6700) at
>> pthread_create.c:312
>> #15 0x00007f547de2a47d in clone () at
>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
>> (gdb) quit
>> 
>> 
>> Contrail-control log during crash
>> -------------------------------------------------
>> 2015-05-09 Sat 14:54:14:617.374 IST  nodei6 [Thread 140000083560192,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh SEND Notification with Code
>> 2 and SubCode 7 ( OPEN Message Error:Unsupported Capability )
>> controller/src/bgp/bgp_session.cc 96
>> 2015-05-09 Sat 14:54:35:205.862 IST  nodei6 [Thread 140000045774592,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  Event: Tcp
>> Connection Closed  peer ip: 192.168.22.4 ( nodei9-1 )
>> controller/src/xmpp/xmpp_state_machine.cc 1322
>> 2015-05-09 Sat 14:54:42:809.939 IST  nodei6 [Thread 140000710457088,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  PassiveOpen
>> in state: Idle peer ip: 192.168.22.4 (  )
>> controller/src/xmpp/xmpp_state_machine.cc 1335
>> 2015-05-09 Sat 14:54:50:564.235 IST  nodei6 [Thread 140000718853888,
>> Pid 18669]: BGP [SYS_WARN]: BgpPeerMessageLog: BGP Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh RECV Unsupported Capability:
>> MpExtension (1) controller/src/bgp/bgp_proto.cc 112
>> 2015-05-09 Sat 14:54:50:564.660 IST  nodei6 [Thread 140000718853888,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh RECV Notification with Code
>> 6 and SubCode 5 ( Cease:Connection is rejected by the peer )
>> controller/src/bgp/state_machine.cc 307
>> 2015-05-09 Sat 14:54:50:564.968 IST  nodei6 [Thread 140000718853888,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh SEND Notification with Code
>> 2 and SubCode 7 ( OPEN Message Error:Unsupported Capability )
>> controller/src/bgp/bgp_session.cc 96
>> 2015-05-09 Sat 14:55:29:444.147 IST  nodei6 [Thread 140000096155392,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  Event: Tcp
>> Connection Closed  peer ip: 192.168.22.5 ( nodei10 )
>> controller/src/xmpp/xmpp_state_machine.cc 1322
>> 2015-05-09 Sat 14:55:31:681.300 IST  nodei6 [Thread 140000731449088,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:nodei8 RECV Notification with Code
>> 6 and SubCode 3 ( Cease:Administrator has unconfigured the peer )
>> controller/src/bgp/state_machine.cc 307
>> 2015-05-09 Sat 14:55:39:548.516 IST  nodei6 [Thread 140000752441088,
>> Pid 18669]: BGP [SYS_WARN]: BgpPeerMessageLog: BGP Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh RECV Unsupported Capability:
>> MpExtension (1) controller/src/bgp/bgp_proto.cc 112
>> 2015-05-09 Sat 14:55:39:548.843 IST  nodei6 [Thread 140000752441088,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh RECV Notification with Code
>> 6 and SubCode 5 ( Cease:Connection is rejected by the peer )
>> controller/src/bgp/state_machine.cc 307
>> 2015-05-09 Sat 14:55:39:549.138 IST  nodei6 [Thread 140000752441088,
>> Pid 18669]: BGP [SYS_NOTICE]: BgpPeerNotificationLog: Bgp Peer
>> default-domain:default-project:ip-fabric:__default__:nodei6:default-domain
>> :default-project:ip-fabric:__default__:walsh SEND Notification with Code
>> 2 and SubCode 7 ( OPEN Message Error:Unsupported Capability )
>> controller/src/bgp/bgp_session.cc 96
>> 2015-05-09 Sat 14:55:39:597.924 IST  nodei6 [Thread 140000714655488,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  PassiveOpen
>> in state: Idle peer ip: 192.168.22.5 (  )
>> controller/src/xmpp/xmpp_state_machine.cc 1335
>> 2015-05-09 Sat 14:55:40:694.229 IST  nodei6 [Thread 140000054171392,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  Event: Tcp
>> Connection Closed  peer ip: 192.168.22.4 ( nodei9 )
>> controller/src/xmpp/xmpp_state_machine.cc 1322
>> 2015-05-09 Sat 14:55:46:241.071 IST  nodei6 [Thread 140000037377792,
>> Pid 18669]: XMPP [SYS_NOTICE]: XmppEventLog:  Mode Server:  Event: Tcp
>> Connection Closed  peer ip: 192.168.22.4 ( nodei9-1 )
>> controller/src/xmpp/xmpp_state_machine.cc 1322
>> 2015-05-09 Sat 14:56:05:916.364 IST  nodei6 [Thread 140012986161088,
>> Pid 31844]: SANDESH: Logging: DISABLED -> ENABLED
>> 2015-05-09 Sat 14:56:05:916.563 IST  nodei6 [Thread 140012986161088,
>> Pid 31844]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_NOTICE ]
>> log4level: [ TRACE ] -> [ WARN ]
>> 2015-05-09 Sat 14:56:05:917.715 IST  nodei6 [Thread 140012986161088,
>> Pid 31844]: Starting Bgp Server at port 179
>> 2015-05-09 Sat 14:56:05:930.579 IST  nodei6 [Thread 140012986161088,
>> Pid 31844]: SANDESH: No Client: 1431163565930447
>> SandeshModuleClientTrace: data= [ name =
>> nodei6:Control:contrail-control:0 client_info= [  status = Idle
>> successful_connections = 0 pid = 31844 http_port = 8083 start_time =
>> 1431163565930223 collector_name =  primary = 0.0.0.0:0 secondary =
>> 0.0.0.0:0 rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0
>> blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =
>> errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0
>> blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =
>> errors = 0 ] ] ]
>> 
>> 
>> 
>> 
>> Testbed
>> --------------
>> 
>> env.roledefs = {
>>     'all': [host1, host2, host3, host4, host5, host6],
>>     'cfgm': [host1, host2, host3],
>>     'openstack': [host1, host2, host3],
>>     'webui': [host2],
>>     'control': [host1, host3],
>>     'compute': [host4, host5, host6],
>>     'tsn': [host4, host5],
>>     'toragent': [host4, host5],
>>     'collector': [host1, host3],
>>     'database': [host1, host2, host3],
>>     'build': [host_build],
>> }
>> 
>> env.hostnames = {
>>     'all': ['nodei6', 'nodei7', 'nodei8', 'nodei9', 'nodei10',
>> 'nodei19']
>> }
>> 
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/juniperopenstack/+bug/1453369/+subscriptions
>