Comment 10 for bug 1593716

Revision history for this message
Ashok Singh (ashoksr) wrote :

We are hitting the issue
https://bugs.launchpad.net/juniperopenstack/+bug/1593716

VM object still present in Agent's config DB. Agent has not sent Unsubscribe for VM bd4466bb-184d-4a9a-981d-6a6302faf2f7. This can happen when there is Xmpp Channel flap because of presence of issue https://bugs.launchpad.net/juniperopenstack/+bug/1593716 in R2.21.2-36

(gdb) p Agent::singleton_->cfg_
$3 = (AgentConfig *) 0x7f9830000d00
(gdb) p $3->vm_cfg_table_
There is no member or method named vm_cfg_table_.
(gdb) p $3->cfg_vm_table_
$4 = (autogen::DBTable_Agent_VirtualMachine *) 0x7f983000c830
(gdb) dump_ifmap_entries 0x7f983000c830
--------------------------------------------
0x7f98281e2f70  name=00256b4b-752e-4c10-a8f6-2f7821e07f81
0x7f98044892c0  name=03e0b056-7d87-43c8-83d8-7f96179a8800
0x7f9814111e60  name=05a4ec9e-544f-4843-a729-b3d4427f30c8
0x7f98343f5bb0  name=0daf5cdc-52ab-4317-a7aa-f1318a373926
0x7f9834398d40  name=18c494c9-61d7-4008-bf13-1dff51d96acb
0x7f98142bee80  name=1fc8ae71-6331-495f-a5ee-e4298ece5725
0x7f9810940b70  name=3d95a1e3-f6d0-4804-8c3c-d7d62b78cfb2
0x7f97fc062b90  name=41d4c09b-84bc-48f2-8e2b-0ea575c98ca0
0x7f98042258e0  name=41f56cb8-3117-4822-8708-c478ebc7b08c
0x7f97f0a59390  name=4c4b151b-9cfe-40d5-8eb2-db7e0b4d1e02
0x7f980032d2a0  name=5ac03053-8049-4508-820d-511c7f365bf2
0x7f980401c5d0  name=842a9d4d-e9f4-4c04-a0b2-2875beeedab1
0x7f98346b0900  name=87d3cc5c-50cc-4c92-bbe1-6cd1140601c1
0x7f981c381c90  name=88e80b6c-c6b7-4068-b501-d118999bb6f8
0x7f981c2f04b0  name=aab86767-ddb6-4728-b7b0-65ee09c2cca2
0x7f982c313d00  name=b8f9616d-035b-4c06-8493-2b4d8d57831c
0x7f983411fa80  name=b9b7a201-83dd-4423-90e7-e29c59c6cfa7
0x7f97fc937340  name=bd4466bb-184d-4a9a-981d-6a6302faf2f7
0x7f97fca69fb0  name=d0b1d2ea-d5fb-4c45-8425-68e805e6a6ea
0x7f983c459e80  name=e5be3c1f-935c-4451-b0a9-80f50049c52d
0x7f9830338130  name=eb23f5d4-0706-429f-8e92-36bf6e7d3983
0x7f9844299800  name=eec8f83f-6ee5-47a5-804c-33b5399c7255
0x7f9811083940  name=f7d53973-0bc4-4c47-884f-7ae0d851bf56

Xmpp Channel flaps confirmed by trace logs of controller taken from core file

controller_trace.log:3220:2017-01-25 09:32:55.881445 AgentXmppSession: peer = "10.3.135.73" event = "NOT_READY" tree_builder = "NULL" message = "BGP peer decommissioned for xmpp channel." file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1465
controller_trace.log:3249:2017-01-25 09:32:55.881880 AgentXmppSession: peer = "10.3.135.72" event = "NOT_READY" tree_builder = "NULL" message = "BGP peer selected asconfig peer on decommission of old config peer." file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1491
controller_trace.log:3252:2017-01-25 09:32:55.881891 AgentXmppSession: peer = "10.3.135.73" event = "NOT_READY" tree_builder = "10.3.135.72" message = "Peer elected Multicast Tree Builder" file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1543
controller_trace.log:3311:2017-01-26 11:26:12.898147 AgentXmppSession: peer = "10.3.135.74" event = "NOT_READY" tree_builder = "NULL" message = "BGP peer decommissioned for xmpp channel." file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1465
controller_trace.log:3667:2017-02-28 23:54:58.624388 AgentXmppSession: peer = "10.3.135.72" event = "NOT_READY" tree_builder = "NULL" message = "BGP peer decommissioned for xmpp channel." file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1465
controller_trace.log:3698:2017-02-28 23:54:58.624927 AgentXmppSession: peer = "10.3.135.73" event = "NOT_READY" tree_builder = "NULL" message = "BGP peer selected asconfig peer on decommission of old config peer." file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1491
controller_trace.log:3701:2017-02-28 23:54:58.624939 AgentXmppSession: peer = "10.3.135.72" event = "NOT_READY" tree_builder = "10.3.135.73" message = "Peer elected Multicast Tree Builder" file = "controller/src/vnsw/agent/controller/controller_peer.cc" line = 1543