On TSN switchover, invalid route update message from backup tsn

Bug #1487567 reported by Vedamurthy Joshi
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Fix Committed
Medium
Manish Singh
Trunk
Fix Committed
Medium
Manish Singh

Bug Description

R2.20 Build 81 Ubuntu 14.04 Juno

On this scale setup with 3 control nodes, when two of the control-nodes were stopped, all the 128 tor-agents established xmpp with single control node.

During this time, it was seen that backup tor-agents were sending these updates ..there were thousands of these...
Prakash and Manish looked into them. Looks like these xmpp messages from backup ta can be avoided altogether

root@nodei34:/var/log/contrail# grep "Bad nexthop address " contrail-control* | wc -l
92341
root@nodei34:/var/log/contrail# grep "Bad inet address " contrail-control* | wc -l
42461
root@nodei34:/var/log/contrail#

2015-08-21 Fri 03:42:00:269.874 PDT nodei34 [Thread 140700964337408, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project9:vn269:vn269 : Bad inet address /32 controller/src/bgp/bgp_xmpp_channel.cc 1457
2015-08-21 Fri 03:42:00:270.432 PDT nodei34 [Thread 140701614434048, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project9:vn270:vn270 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.5:266-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:270.844 PDT nodei34 [Thread 140701614434048, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project9:vn287:vn287 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.5:283-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:271.209 PDT nodei34 [Thread 140701614434048, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project9:vn287:vn287 : Bad inet address /32 controller/src/bgp/bgp_xmpp_channel.cc 1457
2015-08-21 Fri 03:42:00:271.487 PDT nodei34 [Thread 140701614434048, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project9:vn288:vn288 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.5:284-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:271.855 PDT nodei34 [Thread 140701614434048, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.5 in instance default-domain:project7:vn219:vn219 : Bad inet address /32 controller/src/bgp/bgp_xmpp_channel.cc 1457
2015-08-21 Fri 03:42:00:274.731 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.10 in instance default-domain:project9:vn286:vn286 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.10:72-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:275.101 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.10 in instance default-domain:project9:vn286:vn286 : Bad inet address /32 controller/src/bgp/bgp_xmpp_channel.cc 1457
2015-08-21 Fri 03:42:00:275.567 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.10 in instance default-domain:project9:vn287:vn287 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.10:121-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:276.074 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.12 in instance default-domain:project9:vn271:vn271 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.12:267-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:276.478 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.12 in instance default-domain:project9:vn272:vn272 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.12:268-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:276.984 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.12 in instance default-domain:project9:vn272:vn272 : Bad inet address /32 controller/src/bgp/bgp_xmpp_channel.cc 1457
2015-08-21 Fri 03:42:00:277.433 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.12 in instance default-domain:project9:vn273:vn273 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.12:269-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961
2015-08-21 Fri 03:42:00:277.848 PDT nodei34 [Thread 140700981131008, Pid 32673]: BGP [SYS_WARN]: XmppPeerInstanceLog: XMPP Peer nodei34:192.168.1.10 in instance default-domain:project9:vn283:vn283 : Bad nexthop address 0.0.0.0 for multicast route 0-192.168.1.10:270-0.0.0.0,255.255.255.255,0.0.0.0 controller/src/bgp/bgp_xmpp_channel.cc 961

Tags: vrouter
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/13245
Submitter: Manish Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/13245
Committed: http://github.org/Juniper/contrail-controller/commit/36026e54c363878ccbe94ad06a98a491afe7a918
Submitter: Zuul
Branch: R2.20

commit 36026e54c363878ccbe94ad06a98a491afe7a918
Author: Manish <email address hidden>
Date: Mon Aug 24 14:53:51 2015 +0530

TOR agent mcast route updated without active connection.

Tor-agent should export route only when OVS peer has added a path.
By default a local peer path is added irrespective of mode in mcast.
This was to subscribe in TSN mode. However this should not be used for exporting
in Tor agent mode.

Solution is to check for tor agent mode and skip subscription.

Change-Id: Ibc655d90973e5afd5e47826c74c6ff0696c60625
Closes-bug: 1487567

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/13625
Submitter: Manish Singh (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/13625
Committed: http://github.org/Juniper/contrail-controller/commit/562c5a1a5dc0e4f6e6892927f706c1d53ad06d0b
Submitter: Zuul
Branch: master

commit 562c5a1a5dc0e4f6e6892927f706c1d53ad06d0b
Author: Manish <email address hidden>
Date: Mon Aug 24 14:53:51 2015 +0530

TOR agent mcast route updated without active connection.

Tor-agent should export route only when OVS peer has added a path.
By default a local peer path is added irrespective of mode in mcast.
This was to subscribe in TSN mode. However this should not be used for exporting
in Tor agent mode.

Solution is to check for tor agent mode and skip subscription.

Closes-bug: 1487567
(cherry picked from commit 36026e54c363878ccbe94ad06a98a491afe7a918)

Conflicts:
 src/vnsw/agent/ovs_tor_agent/ovsdb_client/test/test_ovs_multicast_local.cc
Change-Id: Ibc655d90973e5afd5e47826c74c6ff0696c60625

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22-dev

Review in progress for https://review.opencontrail.org/13927
Submitter: Vinay Vithal Mahuli (<email address hidden>)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.