Support single-active BGPaaS object attached to multiple VMIs

[Nischal Sheth]

Consider the following BGPaaS use case. There are a number of control + data plane VMs that form a single entity. All VMs participate in the
data plane, but only a single VM is elected to run BGP to the contrail
controller. If the active VM fails, another VM is elected to run BGP.
The source address for the BGPaaS session is a loopback address that
is configured as an AAP on all the VMIs. This common loopback is also
used as the next-hop for all routes advertised by the VM thus allowing
incoming data traffic to be load balanced to all VMs.

Existing BGPaaS functionality requires the creation of a unique BGPaaS
object per VMI, even though there's a single established BGP session at
any given point. This bug tracks an enhancement to allow a single shared
BGPaaS object to be associated with all VMIs for the entity.

The following changes are anticipated:

1. Configuration

Change schema to add a new property to BGPaaS object to indicate that
the object can be associated with multiple VMIs and only a single VMI
can have an established bgp session. The ip address field is mandatory
when this new property is set. A single client bgp-router object will
be created and associated with the BGPaaS object.

2. Control

Control node needs to handle a new incoming tcp session with the same
source port as an existing bgpaas session. It should bring down any
existing bgpaas session and bring up the new one, triggering graceful
restart if appropriate.

3. Vrouter

vRouter and agent need to handle a failure of BGPaaS VMs wherein the
new active VM is on the same vRouter as the previous active VM. IOW,
the newly initiated bgp session will be associated with another VMI
that's on the same vRouter. This may require some tweaks to the logic
to initiate SNAT+DNAT for BGPaaS session. There may also be some tricky
issues here with TCP session setup and TCB cleanup on base OS in control
node since the vROuter uses the same source port for both old and new
sessions.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28962
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/29051
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28962
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[Manish Singh (manishs)]

More information on bug:

BGPaaS port range - Currently default is hard coded in schema transformer to be between 50000 and 50512. Though user can use schema.conf to update it but it would be better to push it to global-vrouter-config. This will help agent in identifying range as well as port start value which will be used to support multiple vmi on same Bgpaaas object.

Solution for supporting multiple VMI on same BgpaaS object:

Schema transformer has a port range reserved for BgpaaS. Say its 50000-50512. For every BGPaaS object created, ST allocates one port from this range. Then this BGPaaS will be linked to multiple VMI, hence all these VMI will use same BGPaaS port. However as mentioned above in bug, CN needs to identify all these sessions uniquely, because of which agent will have to tweak the VMI port for this BGPaaS object.

Port derivation in agent:

Value definitions:
Say there are two BGPaaS object B1(port 50000) and B2(port 50001). B1 is connected to VMI-10 and VMI-11 while B2 is connected to VMI-21 and VMI-22. Also lets define two port ranges- PR and DPR.
PR is the original port range as in ST, so PR = (50,000 - 50,512).
DPR is the derived port range depending on max session supported.
Also assume that max shared session is 5 per BGPaaS.
For each shared VMI agent will allocate a local index, ranging from 0-4(max session = 5).

Deriving port range:
Total_derived_ports = (PR_max - PR_min) * (Max_shared_session - 1)
DPR_min = (PR_min + 1)
DPR_max = (DPR_min + Total_DPR_ports)

Using this
Total_DPR_ports = (50512 - 50000) * (5 - 1) = 2048.
DPR_min = 50513
DPR_max = 52561

Note that if DPR_max exceeds process limit of ports, then bucket should be carved out in reverse manner i.e.
DPR_max = (PR_max - 1)
DPR_min = (DPR_max - Total_DPR_ports)
Hence:
DPR_min = 49999
DPR_max = 47951

Encode Method:
First VMI session to a BGPaaS object will always use the original port given by ST. Any subsequent session on VMI connected to same BGPaaS will use port from DPR.
Say VMI-11 comes up first so VMI-11 will get local-index as 0. Now VMI-12 comes up and agent will see that its the second VMI and hence give local-index as 1.

If VMI_local_index = 0
 then VMI_Port = B1_port
 else VMI_port = (DPR_min + ((B1_port - PR_min) * (Max_shared_session - 1)) + (VMI_local_index - 1))

VMI11 port = 50000 (VMI_local_index is 0).
VMI12 port = (50513 + ((50000 - 50000) * (5 - 1)) + (1 - 1)) = 50513

Now if VMI11 goes off so local index of VMI11 is freed. If any other VMI comes up it will use the first free local index for a BGPaaS object. In this case VMI13 will get 0 and hence use port 50000.

For B2 with port 50001.
VMI21 port = 50001 (VMI_local_index is 0).
VMI22 port = (50513 + ((50001 - 50000) * (5 - 1)) + (1 - 1)) = 50517
VMI23 port = (50513 + ((50001 - 50000) * (5 - 1)) + (2 - 1)) = 50518

Decode Method:
Need to know Max_shared_session, PR_min, PR_max
Using derived port range logic given above, DPR_min and DPR_max can be calculated.

if (port is in range PR_MIN, PR_MAX)
 VMI_port = port
else
 VMI_port = ((port - DPR_min) / (Max_shared_session - 1)) + PR_MIN

VMI11 original = 50000 (falls in PR_min and PR_max)
VMI12 original = ((50513 - 50513) / 4) + ...

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/28962
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/28962
Committed: http://github.com/Juniper/contrail-controller/commit/a35e554b534061a4de05f74efaac537ee1befccf
Submitter: Zuul (<email address hidden>)
Branch: master

commit a35e554b534061a4de05f74efaac537ee1befccf
Author: srinivn <email address hidden>
Date: Mon Feb 20 22:00:31 2017 -0800

Support single-active BGPaaS object attached to multiple VMIs

Closes-Bug: #1649707

Vrouter-Agent changes to support the multiple VMI interfaces for the given
BGPAAS session
- Changes to tweak the BGP service port number.
1. If the given BGP service port is X then the same will be
shifted by the N number of bits to accomodate the additional
VMI interfaces who are sharing this port.
2. It results in the different service port number during the flow
configuration for each VMI interface.
- Changes to configure the number of bits reserved to share
the same service port across VMI interfaces
1. Added new varaible under SERVICE section vrouter-agent.conf file
to configre the number of bits reserved to hold the multiple VMI interfaces.
- Changes in the UT test cases to validate these changes
1. Test cases are added to share the same service port across
multiple VMI interfaces
2. Test cases are added to valdiate the new variable addition to
agent.conf file.

UT Failure Fix :
The changes added to verify the reverse flow in test_bgp_service is not
working when the same test cases are run multiple times. This behavior
is observed irrespective of the new changes. So to proceed further on
this change, we have reverted those reverse flwo checks.

Review Comments Handling :
In the proposed change, have used local index allocator to keep track
the usage of service ports by multiple VMI interfaces. It leads to store
this local index across the reboots of compute nodes and will result in
additional burden to resource manager to maintain one more state to keep
track theses indexes for each service port.
To avoid this additional burden, it is decided to re-use the VMI index
which are allocated by the agent during the interface creation. This
index will be used to identify the each VMI uniquely for the given
service port.
Some of the caveats with this re-use model as follows,
- the reserved shift bits are not directly working with allocatted index,
since both are defined seperately and working independently
- if the VMI index is going beyond the allocated bits space then same
will not be fit in the reserved space, eventhough there is enough
space to store the indexes
Assumption:
- In the case of re-usage of same index during the VMI deletion and
addition, the corresponding flows of deleted VMI interfaces will be
deleted before adding the newer VMI with re-used index.
Have handles all the review comments
- updating the source port with newer source port, since the same will
be used during the flow configuration.
- Retained older source port to identify the users of the given service
port from the config

Review Comments Hanling-II:
After discussion, the logic for the derived port is changed as per the
below comment,
https://bugs.launchpad.net/juniperopenstack/+bug/1649707/comments/17

Change-Id: Ia282380dfc3dd209d3916d8f64bc...

[Nischal Sheth (nsheth)]

Set to InProgress since all changes haven't been committed.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/31999
Submitter: Manish Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/33242
Submitter: Manish Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/31999
Committed: http://github.com/Juniper/contrail-controller/commit/00173ace9848a3bc6c0436f29fe170369ceebaf4
Submitter: Zuul (<email address hidden>)
Branch: master

commit 00173ace9848a3bc6c0436f29fe170369ceebaf4
Author: Manish <email address hidden>
Date: Tue May 23 12:24:49 2017 +0530

Schema changes for bgp as service.

For details, please refer:
https://bugs.launchpad.net/juniperopenstack/+bug/1649707/comments/17

Change-Id: I05b5936a264bb0f619e03a84e5532b2d46a356f6
Closes-bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/33729
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/33862
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/33729
Committed: http://github.com/Juniper/contrail-controller/commit/81bf09257f469712262644063d874dfcc422134c
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit 81bf09257f469712262644063d874dfcc422134c
Author: srinivn <email address hidden>
Date: Tue Jul 18 03:09:53 2017 -0700

Support single-active BGPaaS object attached to multiple VMIs

- updated the review comments
- using IndexVector to allocate index dynamically.
- Added test case for multiple add for same session and vmi

Change-Id: I1dfc3e1205b1be682ed59f484aafdf7bddd23fd8
Closes-Bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34142
Submitter: Ranjeet R (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/33862
Committed: http://github.com/Juniper/contrail-controller/commit/06a5d3e9aa10998a41114731909fc40ee4da917c
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 06a5d3e9aa10998a41114731909fc40ee4da917c
Author: srinivn <email address hidden>
Date: Thu Jul 20 22:12:12 2017 -0700

Support single-active BGPaaS object attached to multiple VMIs

Existing way to derive the extended ports are changed as follows,
- Port range which will be used by bgpaas will be defined as part of
global vrouter config in schema.
- From the given range, the dervied port will be defined as follows,
Ex : 50000 - 50512
Session:1 : config allocates : 50000 for VMI1
Sharing the session-1 for VMI2: allocates 50513
Sharing the session-1 for VMI3: allocates 51026
Session:2 : config allocates : 50001 for VMI4
Sharing the session-2 for VMI5: allocates 50514
Sharing the session-2 for VMI6: allocates 51027
- introduced infra to process global_system config and update bgpaas parameters
- using IndexVector to allcoate index dynamically.
- Added test case for multiple add for same session and vmi

Change-Id: I9c27804e47544abc509b5c9727bacdd683c8ccd1
Closes-Bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34142
Submitter: Ranjeet R (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34733
Submitter: Ranjeet R (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34814
Submitter: Ranjeet R (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34733
Submitter: Ranjeet R (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/34733
Committed: http://github.com/Juniper/contrail-controller/commit/e6f7bfad329a59b0efbfdc5d4f95b93c2b24fc9e
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit e6f7bfad329a59b0efbfdc5d4f95b93c2b24fc9e
Author: Ranjeet R <email address hidden>
Date: Fri Aug 18 15:29:34 2017 -0700

Fixes: Support single-active BGPaaS object attached to multiple VMIs

Changes to schema and API server to accomodate configuring ports
for BGPaaS. Also, changes to make the BGPaaS as a shared service
per VN.

Backporting some changes to GlobalSystemConfig from mainline
to R3.2.

Change-Id: I12bcb24e294ac9a6a1e2e88330c143b4651a53a6
Partial-Bug: 1649707

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/34814
Committed: http://github.com/Juniper/contrail-controller/commit/01186d125aac88cafd7aebd9cbd79ef46e18dd9f
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 01186d125aac88cafd7aebd9cbd79ef46e18dd9f
Author: Ranjeet R <email address hidden>
Date: Fri Jul 28 12:32:29 2017 -0700

Fixes: Support single-active BGPaaS object attached to multiple VMIs

Changes to schema and API server to accomodate configuring
ports for BGPaaS.

Change-Id: I2a6b6ca4e47c714f5b6b846b417648a6c8846219
Partial-Bug: 1649707

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/34142
Committed: http://github.com/Juniper/contrail-controller/commit/19c29e2e66428bd2e988f2c6d37b691258a9bb75
Submitter: Zuul (<email address hidden>)
Branch: master

commit 19c29e2e66428bd2e988f2c6d37b691258a9bb75
Author: Ranjeet R <email address hidden>
Date: Fri Jul 28 12:32:29 2017 -0700

Fixes: Support single-active BGPaaS object attached to multiple VMIs

Changes to schema and API server to accomodate configuring
ports for BGPaaS.

Change-Id: I2a6b6ca4e47c714f5b6b846b417648a6c8846219
Partial-Bug: 1649707

[Ranjeet R (rranjeet-n)]

Schema Transformer related changes are completed and checked into R4.0.1, R3.2.5 and mainline. The bug is still open for Srinivas to make agent code changes for using the bgpaas_shared flag that is pushed by API server to figure out whether BGP router is shared or not.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34981
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/34985
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/34985
Committed: http://github.com/Juniper/contrail-controller/commit/d9d629d292fef0a4d43fea694b6906e090d84f66
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit d9d629d292fef0a4d43fea694b6906e090d84f66
Author: srinivn <email address hidden>
Date: Mon Aug 28 03:57:23 2017 -0700

Support single-active BGPaaS object attached to multiple VMIs

- New flag "bgpaas-shared" is added in the schema to indicate
the given bgp-aas session is shared or not.
- To support this flag in vrouter-agent,
1. If this flag is true, then local index will be allocated for
each VMI for the given source port.
2. If the flag is false, then the session will continue to use
original source port which is given by the controller.

Change-Id: I860f5a2319c2787f35aa0170ddbe2e896dd1efd9
Closes-Bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/34981
Committed: http://github.com/Juniper/contrail-controller/commit/b3aa1a6e845e447cb98300cbc13ec8088297e7ce
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit b3aa1a6e845e447cb98300cbc13ec8088297e7ce
Author: srinivn <email address hidden>
Date: Mon Aug 28 01:58:37 2017 -0700

Support single-active BGPaaS object attached to multiple VMIs

- New flag "bgpaas-shared" is added in the schema to indicate
the given bgp-aas session is shared or not.
- To support this flag in vrouter-agent,
1. If this flag is true, then local index will be allocated for
each VMI for the given source port.
2. If the flag is false, then the session will continue to use
original source port which is given by the controller.

Change-Id: I83eb8a3b16a7561c26a548cd0cfd9799ed666bb5
Closes-Bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/35179
Submitter: Srinivasan Venkatakrishnan (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/35179
Committed: http://github.com/Juniper/contrail-controller/commit/04077096ba3a1d89a233c0086647f21d315b547f
Submitter: Zuul (<email address hidden>)
Branch: master

commit 04077096ba3a1d89a233c0086647f21d315b547f
Author: srinivn <email address hidden>
Date: Fri Sep 1 03:38:58 2017 -0700

Support single-active BGPaaS object attached to multiple VMIs

- New flag "bgpaas-shared" is added in the schema to indicate
the given bgp-aas session is shared or not.
- To support this flag in vrouter-agent,
1. If this flag is true, then local index will be allocated for
each VMI for the given source port.
2. If the flag is false, then the session will continue to use
original source port which is given by the controller.

Change-Id: I6fc4c7e00c2e0a112d5855f9692870d1664d0983
Closes-Bug: #1649707

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review2.opencontrail.org/34142
Submitter: Krzysztof Klimonda (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review2.opencontrail.org/34142
Committed: http://github.com/Juniper/contrail-controller/commit/19c29e2e66428bd2e988f2c6d37b691258a9bb75
Submitter: Krzysztof Klimonda (<email address hidden>)
Branch: master

commit 19c29e2e66428bd2e988f2c6d37b691258a9bb75
Author: Ranjeet R <email address hidden>
Date: Fri Jul 28 12:32:29 2017 -0700

Fixes: Support single-active BGPaaS object attached to multiple VMIs

Changes to schema and API server to accomodate configuring
ports for BGPaaS.

Change-Id: I2a6b6ca4e47c714f5b6b846b417648a6c8846219
Partial-Bug: 1649707