one of xmpp sessions not setup on repeated control-node restarts

Bug #1527844 reported by Vedamurthy Joshi
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Won't Fix
High
Nipa
Trunk
New
High
Vedamurthy Joshi

Bug Description

R2.20 Build 115 (with vrouter code same as latest R2.20 code)

A setup similar to bug 1527551, but I made sure config scripts stops so that control node becomes active

Control node on nodec1(10.204.216.58), nodec3(10.204.216.60) became active.
But on one of the compute nodes nodek2, the xmpp session did not become active
gcores of control nodes and agent will be in http://10.204.216.50/Docs/bugs/#

root@nodek2:~# contrail-status
== Contrail vRouter ==
supervisor-vrouter: active
contrail-vrouter-agent initializing (XMPP:control-node:10.204.216.58, XMPP:control-node:10.204.216.60 connection down)
contrail-vrouter-nodemgr active

root@nodec1:~# contrail-status | grep control
supervisor-control: active
contrail-control active
contrail-control-nodemgr active
root@nodec1:~#

root@nodec3:/var/log/contrail# contrail-status | grep control
supervisor-control: active
contrail-control active
contrail-control-nodemgr active
root@nodec3:/var/log/contrail#
======================================

root@nodec3:/var/log/contrail# tail contrail-control.log
2015-12-19 Sat 01:27:58:471.173 IST nodec3 [Thread 139778180577024, Pid 14238]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Server: PassiveOpen in state: Idle peer ip: 10.204.216.222 ( ) controller/src/xmpp/xmpp_state_machine.cc 1341
2015-12-19 Sat 01:27:58:471.940 IST nodec3 [Thread 139778180577024, Pid 14238]: BGP [SYS_NOTICE]: BgpMessageLog: Peer not found on channel not ready event controller/src/bgp/bgp_xmpp_channel.cc 2407
2015-12-19 Sat 01:28:04:314.249 IST nodec3 [Thread 139778100033280, Pid 14238]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Server: PassiveOpen in state: Idle peer ip: 10.204.216.222 ( ) controller/src/xmpp/xmpp_state_machine.cc 1341
2015-12-19 Sat 01:28:04:315.538 IST nodec3 [Thread 139778100033280, Pid 14238]: BGP [SYS_NOTICE]: BgpMessageLog: Peer not found on channel not ready event controller/src/bgp/bgp_xmpp_channel.cc 2407
2015-12-19 Sat 01:28:26:402.774 IST nodec3 [Thread 139778100033280, Pid 14238]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Server: PassiveOpen in state: Idle peer ip: 10.204.216.222 ( ) controller/src/xmpp/xmpp_state_machine.cc 1341
2015-12-19 Sat 01:28:26:403.874 IST nodec3 [Thread 139778180577024, Pid 14238]: BGP [SYS_NOTICE]: BgpMessageLog: Peer not found on channel not ready event controller/src/bgp/bgp_xmpp_channel.cc 2407
2015-12-19 Sat 01:28:29:252.962 IST nodec3 [Thread 139778100033280, Pid 14238]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Server: PassiveOpen in state: Idle peer ip: 10.204.216.222 ( ) controller/src/xmpp/xmpp_state_machine.cc 1341
2015-12-19 Sat 01:28:29:253.663 IST nodec3 [Thread 139778100033280, Pid 14238]: BGP [SYS_NOTICE]: BgpMessageLog: Peer not found on channel not ready event controller/src/bgp/bgp_xmpp_channel.cc 2407
2015-12-19 Sat 01:28:35:368.260 IST nodec3 [Thread 139778176378624, Pid 14238]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Server: PassiveOpen in state: Idle peer ip: 10.204.216.222 ( ) controller/src/xmpp/xmpp_state_machine.cc 1341
2015-12-19 Sat 01:28:35:368.625 IST nodec3 [Thread 139778176378624, Pid 14238]: BGP [SYS_NOTICE]: BgpMessageLog: Peer not found on channel not ready event controller/src/bgp/bgp_xmpp_channel.cc 2407
=============================

root@nodek2:/var/log/contrail# tail contrail-vrouter-agent.log
2015-12-19 Sat 14:35:42:209.707 IST nodek2 [Thread 139688113178368, Pid 22667]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Client: Event: Tcp Connected peer ip: 10.204.216.59 ( <email address hidden> ) controller/src/xmpp/xmpp_state_machine.cc 1313
2015-12-19 Sat 14:35:42:210.452 IST nodek2 [Thread 139688113178368, Pid 22667]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Client: Event: Tcp Connection Closed peer ip: 10.204.216.59 ( <email address hidden> ) controller/src/xmpp/xmpp_state_machine.cc 1328
2015-12-19 Sat 14:35:43:867.153 IST nodek2 [Thread 139688117376768, Pid 22667]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Client: Event: Tcp Connected peer ip: 10.204.216.58 ( <email address hidden> ) controller/src/xmpp/xmpp_state_machine.cc 1313
2015-12-19 Sat 14:35:43:867.807 IST nodek2 [Thread 139688117376768, Pid 22667]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Client: Event: Tcp Connection Closed peer ip: 10.204.216.58 ( <email address hidden> ) controller/src/xmpp/xmpp_state_machine.cc 1328
2015-12-19 Sat 14:35:50:676.920 IST nodek2 [Thread 139688138368768, Pid 22667]: XMPP [SYS_NOTICE]: XmppEventLog: Mode Client: Event: Tcp Connected peer ip: 10.204.216.59 ( <email address hidden> ) controller/src/xmpp/xmpp_state_machine.cc 1313

=========================

xmpp status from control node though shows status as Established :
view-source:http://10.204.216.58:8083/Snh_ShowXmppConnectionReq

<?xml-stylesheet type="text/xsl" href="/universal_parse.xsl"?><ShowXmppConnectionResp type="sandesh"><connections type="list" identifier="1"><list type="struct" size="6"><ShowXmppConnection><name type="string" identifier="1">nodek1</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.216.221:35119</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:37:39.040382</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection><ShowXmppConnection><name type="string" identifier="1">nodek2</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.216.222:56367</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:35:18.974261</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection><ShowXmppConnection><name type="string" identifier="1">nodek3-2</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.216.223:36685</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:35:27.152544</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection><ShowXmppConnection><name type="string" identifier="1">nodeg11-3</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.217.51:38405</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:38:12.825778</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection><ShowXmppConnection><name type="string" identifier="1">nodeg11-4</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.217.51:42292</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:35:18.995822</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection><ShowXmppConnection><name type="string" identifier="1">nodeg11</name><deleted type="bool" identifier="2">false</deleted><remote_endpoint type="string" identifier="3">10.204.217.51:44146</remote_endpoint><local_endpoint type="string" identifier="4">10.204.216.58:5269</local_endpoint><state type="string" identifier="5">Established</state><last_event type="string" identifier="6">xmsm::EvXmppKeepalive</last_event><last_state type="string" identifier="7">Active</last_state><last_state_at type="string" identifier="8">2015-Dec-18 19:50:40.499301</last_state_at><receivers type="list" identifier="9"><list type="string" size="2"><element>IFMap</element><element>BGP</element></list></receivers></ShowXmppConnection></list></connections><more type="bool" identifier="0">false</more></ShowXmppConnectionResp>

Tags: xmpp
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.