FAt flow enhancements - configurable prefix-lenghth - for flow creation

[richard roberts]

This LP works in conjunction with other LP for fat flow enhancements (LP 1739544 and LP ???)

Flow setup rate is challenge in SP designs with millions of subscribers. this is especially true when dealing with failure scenarios where (fast) reconvergence from one compute to another is requested

The UC assumes a typical internet access UC for residential services (broadband or mobile).
In this case, most of the connectivity(flows) is setup from SP subsrcibers to the Internet.
 - SP subcsribers have known IP address pool (say 10/8)
 - Internet has many

We assume then that first packet has SRC in the SP pool (here 10/8).

LP 1739544 provides the ability to boost the ramp up rate of contrail by creating flows based on IP SRC only.

This a great leap forward in term of perfs and DDOS protection (1 flow for any type of DEST), however this may still be challenging for some UC where subsecond convergence is requested.

This LP aims at improving the ramp-up by grouping contiguous sources in same flows with a configurable prefix-length

In our example, 10/8 (subscriber pool) is split into smaller pools example 10/28
10.0.0.1, 10.0.0.2, 10.0.0.3 … 10.0.0.15 are in a same flow

Entropy is still good enough for Load balancing to operate say 1M subcribers => 1M/16 flows .

Flow setup is rate is proportional to the aggregation achieved with the prefixl-length. In this example, flow setup rate is multiplied by 16 (=/28)

Caveat: packet steering is not possible wihtout additional tweaking (see LP ???).

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/42495
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/42754
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/42495
Submitter: Ashok Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/42495
Committed: http://github.com/Juniper/contrail-api-client/commit/ccb787d0866c7e108ade91297ab5774dd5c2b5bf
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit ccb787d0866c7e108ade91297ab5774dd5c2b5bf
Author: Ashok Singh R <email address hidden>
Date: Wed Apr 25 12:16:34 2018 +0530

Update fat-flow configuration in schema

Define source-prefix and destination-prefix options in FatFlow
configuration

Change-Id: Ib61dadd6b125bb0e52cf77053371d34d83ef2b14
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/42754
Committed: http://github.com/Juniper/contrail-api-client/commit/b7dfc67f50414da802da0a66562d824bc7a7e103
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit b7dfc67f50414da802da0a66562d824bc7a7e103
Author: Ashok Singh R <email address hidden>
Date: Wed Apr 25 12:16:34 2018 +0530

Update fat-flow configuration in schema

Define source-prefix and destination-prefix options in FatFlow
configuration

Partial-Bug: #1752618
(cherry picked from commit 25be0e4285b6aed8d5803f1c27a80196a28c173a)

Change-Id: I54fd875a9eb0715b5bbe8edceb8ea658f4bda1ba

[Raghunandan Srinivasan (raghus)]

will be targeted for R5.1

[Raghunandan Srinivasan (raghus)]

After revisiting the issue with design discussions, we have decided that it can make it R5.0.2

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45668
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45809
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45668
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45809
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45809
Committed: http://github.com/Juniper/contrail-api-client/commit/3da3a56da75ce9f232c1e532e81d7c46502ba136
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 3da3a56da75ce9f232c1e532e81d7c46502ba136
Author: Anand Narayanan Rao <email address hidden>
Date: Tue Aug 28 15:43:02 2018 +0530

Schema changes for fat flow enhancments for R5.1.
- Add src-aggregate-prefix-length and dst-aggregate-prefix-length

Change-Id: Id1f5849525b049f238e6607e67841f737ad8d3d6
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45992
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45992
Committed: http://github.com/Juniper/contrail-controller/commit/fce65a4ce2666364c5f7786cf36ca6b7723eb104
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit fce65a4ce2666364c5f7786cf36ca6b7723eb104
Author: Anand Narayanan Rao <email address hidden>
Date: Wed Sep 5 14:48:12 2018 +0530

Initial agent changes for fat flow enhancements for R5.1
- Data structure changes
- Validation of config
- Storing config in VmInterface object
- Unit test cases

Change-Id: Ie84e5ff38253aa4b53a86bda8d7d91b74951ea22
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45668
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/47470
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/47471
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/47470
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/47471
Committed: http://github.com/Juniper/contrail-vrouter/commit/728661785afa9476eeb6ec8ea08b65e8be78d8a7
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 728661785afa9476eeb6ec8ea08b65e8be78d8a7
Author: Anand Narayanan Rao <email address hidden>
Date: Thu Nov 15 22:48:28 2018 +0530

Implementation of prefix based fat flows for R5.1 - Vrouter changes.
More details of the feature are available @
https://review.opencontrail.org/#/c/45668/

UT done with different fat flow rule combinations.

NOTE: This is part1 of the diff and doesn't include flow key changes to include
prefix mask as part of the flow key.

Change-Id: Ide2be55aa9e43801654088414b86ce9f0dc28af6
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/47470
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/47726
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/47470
Committed: http://github.com/Juniper/contrail-controller/commit/3d5cb32917cabc4581bfcc2443ee0a7b53c2d855
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 3d5cb32917cabc4581bfcc2443ee0a7b53c2d855
Author: Anand Narayanan Rao <email address hidden>
Date: Fri Nov 16 01:11:14 2018 +0530

Implementation of prefix based fat flows for R5.1 - Agent changes.
More details of the feature are available @
https://review.opencontrail.org/#/c/45668/

UT done with different fat flow rule combinations.

NOTE: This is part1 of the diff and doesn't include flow key changes to include
prefix mask as part of the flow key.

Change-Id: Iea497a336ce8ba6b493b178b3172eee946cee406
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45668
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/47726
Committed: http://github.com/Juniper/contrail-vrouter/commit/d4e478c150d792b5fde618237ab3a33345417108
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit d4e478c150d792b5fde618237ab3a33345417108
Author: Anand Narayanan Rao <email address hidden>
Date: Mon Nov 19 15:59:28 2018 +0530

Bugfix: 5.1 prefix based fat flows

In case ignore src/dst is configured, the fat flow src ip/dst ip
mask is used to mask the IP. And based on whether the pkt is coming
from vif or fab the src and dst ip mask is fixed.
Bug: The mask was getting modified based on direction in 2 places,
once outside the api which calls the fat flow lookup api,
and 2nd time inside the fat flow lookup api. This lead to setting
up the mask wrongly.
Fix: Check for direction only in the outside api and not within the
fat flow lookup api.

Change-Id: I034999e3ce93bc32610ae9a64b6f2a8a96d88c6f
Partial-Bug: #1752618

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45668
Submitter: N Anand Rao (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45668
Committed: http://github.com/Juniper/contrail-specs/commit/e4b564a739dab3cf6f17d8cdd8bf068300a810de
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit e4b564a739dab3cf6f17d8cdd8bf068300a810de
Author: Anand Narayanan Rao <email address hidden>
Date: Mon Aug 20 22:12:30 2018 +0530

Fat flow enhancements for R5.1

This blueprint describes the following new fat flow enhancements:-
1) Support for Fat flow exclude prefix list per VMI (Added as part of
R5.0.1).
This is an internal feature which solves some of the issues with
BGPaaS and SSH to metadata IPs in conjunction with fat flow.

2) Support for configurable prefix length when creating fat flows
(R5.1).
For eg: To be able to group flows from 10/8 subnet into groups of
10/28 fat flows.
This is conducive for conserving flow resources in certain SP
usecases who may have a large number of subscribers.

Change-Id: Ica301c4eee0ddfad04819b31759a194c6b35eca3
Partial-Bug: #1752618