Juniper Openstack

vnswad crashed after setup all in virtual testbed

Bug #1328843 reported by Sandip Dey on 2014-06-11

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Juniper Openstack	Fix Committed	Undecided	Hari Prasad Killi

Bug Description

Build:Mainline 2209

Was trying to bring up virtual testbed.
Had 4 control nodes and 10 compute nodes
Vnswad crashed in one of the compute nodes

logs saved at /cs-shared/bugs/<bug-id> on any blr shell server

BT
====
#0 0x00000000009ceae0 in AgentDnsXmppChannel::HandleXmppClientChannelEvent(AgentDnsXmppChannel*, xmps::PeerState) ()
#1 0x00000000009baaf4 in boost::detail::function::void_function_obj_invoker2<boost::_bi::bind_t<void, void (*)(AgentDnsXmppChannel*, xmps::PeerState), boost::_bi::list2<boost::_bi::value<AgentDnsXmppChannel*>, boost::arg<2> > >, void, XmppChannelMux*, xmps::PeerState>::invoke(boost::detail::function::function_buffer&, XmppChannelMux*, xmps::PeerState) ()
#2 0x0000000000aae338 in XmppClient::NotifyConnectionEvent(XmppChannelMux*, xmps::PeerState) ()
#3 0x0000000000ade53c in xmsm::XmppStreamEstablished::XmppStreamEstablished(boost::statechart::state<xmsm::XmppStreamEstablished, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::my_context) ()
#4 0x0000000000ad6861 in ?? ()
#5 0x0000000000adf632 in xmsm::OpenSent::react(xmsm::EvXmppOpen const&) ()
#6 0x0000000000ae5390 in boost::statechart::detail::reaction_result boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::local_react_impl_non_empty::local_react_impl<boost::mpl::list3<boost::statechart::custom_reaction<xmsm::EvXmppOpen>, boost::statechart::custom_reaction<xmsm::EvHoldTimerExpired>, boost::statechart::custom_reaction<xmsm::EvStop> >, boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0> >(boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>&, boost::statechart::event_base const&, void const*) ()
#7 0x0000000000ae546e in boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*) ()
#8 0x0000000000ae49ab in boost::statechart::state_machine<XmppStateMachine, xmsm::Idle, std::allocator<void>, boost::statechart::null_exception_translator>::send_event(boost::statechart::event_base const&) ()
#9 0x0000000000ad9fd1 in ?? ()
#10 0x0000000000ada466 in XmppStateMachine::DequeueEvent(boost::intrusive_ptr<boost::statechart::event_base const>&) ()
#11 0x0000000000ae4f54 in QueueTaskRunner<boost::intrusive_ptr<boost::statechart::event_base const>, WorkQueue<boost::intrusive_ptr<boost::statechart::event_base const> > >::RunQueue() ()
#12 0x0000000000d34d10 in TaskImpl::execute() ()

See original description

Tags:

Hari Prasad Killi (haripk) on 2014-06-19

Changed in juniperopenstack:
assignee:	nobody → Hari Prasad Killi (haripk)

Hari Prasad Killi (haripk) on 2014-06-25

tags:

added: vnsw-agent

Revision history for this message

venkata surya narayana raju datla (vdatla) wrote on 2014-07-09:

Download full text (8.3 KiB)

complete BT:

complete BT:

Core was generated by `/usr/bin/vnswad'.
Program terminated with signal 11, Segmentation fault.
#0  AgentDnsXmppChannel::HandleXmppClientChannelEvent (peer=0x7f882c004690, state=xmps::READY) at controller/src/vnsw/agent/controller/controller_dns.cc:75
75      controller/src/vnsw/agent/controller/controller_dns.cc: No such file or directory.
(gdb) bt
#0  AgentDnsXmppChannel::HandleXmppClientChannelEvent (peer=0x7f882c004690, state=xmps::READY) at controller/src/vnsw/agent/controller/controller_dns.cc:75
#1  0x00000000009baaf4 in operator()<void (*)(AgentDnsXmppChannel*, xmps::PeerState), boost::_bi::list2<XmppChannelMux*&, xmps::PeerState&> > (a=<synthetic pointer>, this=<optimized out>,
    f=<optimized out>) at build/include/boost/bind/bind.hpp:313
#2  operator()<XmppChannelMux*, xmps::PeerState> (this=<optimized out>, a1=<optimized out>, a2=<optimized out>) at build/include/boost/bind/bind_template.hpp:61
#3  boost::detail::function::void_function_obj_invoker2<boost::_bi::bind_t<void, void (*)(AgentDnsXmppChannel*, xmps::PeerState), boost::_bi::list2<boost::_bi::value<AgentDnsXmppChannel*>, boost::arg<2> > >, void, XmppChannelMux*, xmps::PeerState>::invoke (function_obj_ptr=..., a0=0x7f882c003cd0, a1=<optimized out>) at build/include/boost/function/function_template.hpp:153
#4  0x0000000000aae338 in operator() (a1=xmps::READY, a0=0x7f882c003cd0, this=0x7f883ff62620) at build/include/boost/function/function_template.hpp:763
#5  XmppClient::NotifyConnectionEvent (this=0x7f882c0028c0, mux=0x7f882c003cd0, state=xmps::READY) at controller/src/xmpp/xmpp_client.cc:159
#6  0x0000000000ade53c in xmsm::XmppStreamEstablished::XmppStreamEstablished (this=<optimized out>, ctx=...) at controller/src/xmpp/xmpp_state_machine.cc:637
#7  0x0000000000ad6861 in shallow_construct (outermostContextBase=..., pContext=<optimized out>) at build/include/boost/statechart/state.hpp:89
#8  deep_construct (outermostContextBase=..., pContext=<optimized out>) at build/include/boost/statechart/state.hpp:79
#9  boost::statechart::detail::inner_constructor<boost::mpl::l_item<mpl_::long_<1l>, xmsm::XmppStreamEstablished, boost::mpl::l_end>, boost::statechart::state_machine<XmppStateMachine, xmsm::Idle, std::allocator<void>, boost::statechart::null_exception_translator> >::construct (outermostContextBase=..., pContext=<optimized out>) at build/include/boost/statechart/detail/constructor.hpp:93
#10 0x0000000000adf632 in transit_impl<xmsm::XmppStreamEstablished, XmppStateMachine, boost::statechart::detail::no_transition_function> (this=0x7f8838020a50, transitionAction=...)
    at build/include/boost/statechart/simple_state.hpp:798
#11 transit<xmsm::XmppStreamEstablished> (this=0x7f8838020a50) at build/include/boost/statechart/simple_state.hpp:314
#12 xmsm::OpenSent::react (this=0x7f8838020a50, event=...) at controller/src/xmpp/xmpp_state_machine.cc:498
#13 0x0000000000ae5390 in react<xmsm::OpenSent, boost::statechart::event_base, void const*> (evt=..., stt=..., eventType=<optimized out>) at build/include/boost/statechart/custom_reaction.hpp:42
#14 boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::local_react_impl_non_empty::local_react_impl<boost::mpl::list3<boost::statechart::custom_reaction<xmsm::EvXmppOpen>, boost::statechart::custom_reaction<xmsm::EvHoldTimerExpired>, boost::statechart::custom_reaction<xmsm::EvStop> >, boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0> > (stt=..., evt=..., eventType=0x11ba400) at build/include/boost/statechart/simple_state.hpp:816
#15 0x0000000000ae546e in local_react<boost::mpl::list3<boost::statechart::custom_reaction<xmsm::EvXmppOpen>, boost::statechart::custom_reaction<xmsm::EvHoldTimerExpired>, boost::statechart::custom_reaction<xmsm::EvStop> > > (eventType=0x11ba400, evt=..., this=0x7f8838020a50) at build/include/boost/statechart/simple_state.hpp:851
#16 local_react_impl<boost::mpl::list<boost::statechart::custom_reaction<xmsm::EvTcpClose>, boost::statechart::custom_reaction<xmsm::EvXmppOpen>, boost::statechart::custom_reaction<xmsm::EvHoldTimerExpired>, boost::statechart::custom_reaction<xmsm::EvStop> >, boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0> > (
    stt=..., eventType=0x11ba400, evt=...) at build/include/boost/statechart/simple_state.hpp:820
#17 local_react<boost::mpl::list<boost::statechart::custom_reaction<xmsm::EvTcpClose>, boost::statechart::custom_reaction<xmsm::EvXmppOpen>, boost::statechart::custom_reaction<xmsm::EvHoldTimerExpired>, boost::statechart::custom_reaction<xmsm::EvStop> > > (eventType=0x11ba400, evt=..., this=0x7f8838020a50) at build/include/boost/statechart/simple_state.hpp:851
#18 boost::statechart::simple_state<xmsm::OpenSent, XmppStateMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl (this=0x7f8838020a50, evt=..., eventType=0x11ba400)
    at build/include/boost/statechart/simple_state.hpp:489
#19 0x0000000000ae49ab in operator() (this=<synthetic pointer>) at build/include/boost/statechart/state_machine.hpp:87
#20 operator()<boost::statechart::detail::send_function<boost::statechart::detail::state_base<std::allocator<void>, boost::statechart::detail::rtti_policy>, boost::statechart::event_base, const void*>, boost::statechart::state_machine<XmppStateMachine, xmsm::Idle>::exception_event_handler> (action=..., this=<optimized out>) at build/include/boost/statechart/null_exception_translator.hpp:33
#21 boost::statechart::state_machine<XmppStateMachine, xmsm::Idle, std::allocator<void>, boost::statechart::null_exception_translator>::send_event (this=0x7f882c0031e0, evt=...)
    at build/include/boost/statechart/state_machine.hpp:897
#22 0x0000000000ad9fd1 in process_event (evt=..., this=0x7f882c0031e0) at build/include/boost/statechart/state_machine.hpp:279
#23 XmppStateMachine::DequeueEvent (this=0x7f882c0031e0, event=...) at controller/src/xmpp/xmpp_state_machine.cc:1068
#24 0x0000000000ada466 in delete_object<boost::statechart::event_base> (pObject=<optimized out>) at build/include/boost/statechart/event_base.hpp:38
#25 intrusive_ptr_release (pBase=0x7f8830080f30) at build/include/boost/statechart/event_base.hpp:91
#26 ~intrusive_ptr (this=<optimized out>, __in_chrg=<optimized out>) at build/include/boost/smart_ptr/intrusive_ptr.hpp:96
#27 reset (this=0x7f883ff62c60) at build/include/boost/smart_ptr/intrusive_ptr.hpp:140
#28 XmppStateMachine::DequeueEvent (this=0x7f882c0031e0, event=...) at controller/src/xmpp/xmpp_state_machine.cc:1060
#29 0x0000000000ae4f54 in operator() (a0=..., this=0x7f883ff62c30) at build/include/boost/function/function_template.hpp:763
#30 QueueTaskRunner<boost::intrusive_ptr<boost::statechart::event_base const>, WorkQueue<boost::intrusive_ptr<boost::statechart::event_base const> > >::RunQueue (this=0x7f883006eba0)
    at controller/src/base/queue_task.h:53
#31 0x0000000000d34d10 in TaskImpl::execute (this=0x7f8830068040) at controller/src/base/task.cc:224
#32 0x00007f8841e4aece in ?? () from /usr/lib/libtbb_debug.so.2
---Type <return> to continue, or q <return> to quit---
#33 0x00007f8841e41e0b in ?? () from /usr/lib/libtbb_debug.so.2
#34 0x00007f8841e406f2 in ?? () from /usr/lib/libtbb_debug.so.2
#35 0x00007f8841e3b3ce in ?? () from /usr/lib/libtbb_debug.so.2
#36 0x00007f8841e3b270 in ?? () from /usr/lib/libtbb_debug.so.2
#37 0x00007f8841992e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#38 0x00007f8840ead3fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#39 0x0000000000000000 in ?? ()
(gdb)

Ashish Ranjan (aranjan-n) on 2014-07-16

information type:

Proprietary → Public

Revision history for this message

Manish Singh (manishs) wrote on 2014-07-21:

Fixed under
https://github.com/Juniper/contrail-controller/commit/41f9f5a4d10b6c04936c7740c7a425c50169e826