Lost FIP connectivity when schema restarting

Bug #1418701 reported by Édouard Thuleau
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Fix Committed
High
Sachin Bansal
R1.1
Fix Committed
High
Sachin Bansal
R2.0
Fix Committed
High
Sachin Bansal
R2.1
Fix Committed
High
Sachin Bansal
OpenContrail
Fix Committed
High
Édouard Thuleau

Bug Description

On release 1.10, we lost connectivity with FIP when we restart the schema transformer daemon when it walk the IF-MAP graph.
Our control nodes are peering with MX routers and we defined specific route targets (64516:1, 223.255.254.0:1, 223.255.254.0:255) on the floating IP pool virtual networks (called 'public').

From the control node trace, we saw the route targets on the virtual network are updated:

2015-02-05 10:30:46.500 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:223.255.254.0:1, (_iter74) = target:64516:1, (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:223.255.254.0:1, (_iter75) = target:64516:1, (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334
2015-02-05 10:30:46.543 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:223.255.254.0:1, (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:223.255.254.0:1, (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334
2015-02-05 10:30:46.588 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334
2015-02-05 10:30:46.631 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334
2015-02-05 10:31:26.330 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:223.255.254.0:1, (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:223.255.254.0:1, (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334
2015-02-05 10:31:26.392 RoutingInstanceUpdate: str6 = Routing Instance name = default-domain:aaa:public:public str7 = Import Targets import_rt= [ [ (_iter74) = target:223.255.254.0:1, (_iter74) = target:223.255.254.0:255, (_iter74) = target:64516:1, (_iter74) = target:64516:8000001, ] ] str8 = Export Targets export_rt= [ [ (_iter75) = target:223.255.254.0:1, (_iter75) = target:223.255.254.0:255, (_iter75) = target:64516:1, (_iter75) = target:64516:8000001, ] ] str9 = Virtual Network virtual_network = default-domain:aaa:public str10 = VN Index index = 38 file = controller/src/bgp/routing-instance/routing_instance.cc line = 334

That the cause the FIP connectivity goes down and not the east-west traffic.

Tags: config
description: updated
Pedro Marques (5-roque)
Changed in juniperopenstack:
assignee: nobody → Sachin Bansal (sbansal)
Changed in opencontrail:
assignee: nobody → Sachin Bansal (sbansal)
Changed in juniperopenstack:
assignee: Sachin Bansal (sbansal) → Édouard Thuleau (ethuleau)
Changed in opencontrail:
assignee: Sachin Bansal (sbansal) → Édouard Thuleau (ethuleau)
Revision history for this message
Édouard Thuleau (ethuleau) wrote :
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/7326
Committed: http://github.org/Juniper/contrail-controller/commit/28a871d77461175b0b5e76eb9602608df3062ec5
Submitter: Zuul
Branch: master

commit 28a871d77461175b0b5e76eb9602608df3062ec5
Author: Sachin Bansal <email address hidden>
Date: Wed Feb 11 11:16:34 2015 -0800

Do not remove configured RT from RI on restart

When Schema Transformer restarts, it currently resets all route targets on all
routing instances. If there were any RTs configured on the parent VN, those will
be removed and added back when notification is received for adding those targets
to the VN. This can cause lack of connectivity for some time. Instead, now we
read the configured targets when routing istance is created and add the
configured route targets to the RI.

Change-Id: Id9bdb45c5a782bd4e4ae03a29dd1cd37ca20b081
Closes-Bug: 1418701

Changed in juniperopenstack:
status: New → Fix Committed
Changed in opencontrail:
status: New → Fix Committed
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/7367
Committed: http://github.org/Juniper/contrail-controller/commit/c78cfb34e2d018234212c3b07f56eeff78eef23f
Submitter: Zuul
Branch: R2.1

commit c78cfb34e2d018234212c3b07f56eeff78eef23f
Author: Sachin Bansal <email address hidden>
Date: Wed Feb 11 11:16:34 2015 -0800

Do not remove configured RT from RI on restart

When Schema Transformer restarts, it currently resets all route targets on all
routing instances. If there were any RTs configured on the parent VN, those will
be removed and added back when notification is received for adding those targets
to the VN. This can cause lack of connectivity for some time. Instead, now we
read the configured targets when routing istance is created and add the
configured route targets to the RI.

Change-Id: Id9bdb45c5a782bd4e4ae03a29dd1cd37ca20b081
Closes-Bug: 1418701
(cherry picked from commit 28a871d77461175b0b5e76eb9602608df3062ec5)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/7370
Committed: http://github.org/Juniper/contrail-controller/commit/3ffbb76418d08a9b10a566723f6ec7fe64dab9e1
Submitter: Zuul
Branch: R2.0

commit 3ffbb76418d08a9b10a566723f6ec7fe64dab9e1
Author: Sachin Bansal <email address hidden>
Date: Wed Feb 11 11:16:34 2015 -0800

Do not remove configured RT from RI on restart

When Schema Transformer restarts, it currently resets all route targets on all
routing instances. If there were any RTs configured on the parent VN, those will
be removed and added back when notification is received for adding those targets
to the VN. This can cause lack of connectivity for some time. Instead, now we
read the configured targets when routing istance is created and add the
configured route targets to the RI.

Change-Id: Id9bdb45c5a782bd4e4ae03a29dd1cd37ca20b081
Closes-Bug: 1418701
(cherry picked from commit 28a871d77461175b0b5e76eb9602608df3062ec5)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/7363
Committed: http://github.org/Juniper/contrail-controller/commit/0adc10ac2b12f1461ab5db61a5f4b5372045f58f
Submitter: Zuul
Branch: R1.10

commit 0adc10ac2b12f1461ab5db61a5f4b5372045f58f
Author: Sachin Bansal <email address hidden>
Date: Wed Feb 11 11:16:34 2015 -0800

Do not remove configured RT from RI on restart

When Schema Transformer restarts, it currently resets all route targets on all
routing instances. If there were any RTs configured on the parent VN, those will
be removed and added back when notification is received for adding those targets
to the VN. This can cause lack of connectivity for some time. Instead, now we
read the configured targets when routing istance is created and add the
configured route targets to the RI.

Closes-Bug: 1418701
(cherry picked from commit 28a871d77461175b0b5e76eb9602608df3062ec5)

Conflicts:
 src/config/schema-transformer/to_bgp.py
Change-Id: Id9bdb45c5a782bd4e4ae03a29dd1cd37ca20b081

Changed in juniperopenstack:
importance: Undecided → High
Changed in opencontrail:
importance: Undecided → High
Changed in juniperopenstack:
assignee: Édouard Thuleau (ethuleau) → Sachin Bansal (sbansal)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.