Support large number of routing instances

[Nischal Sheth]

Tracking bug to make CN support a very large number of routing instances.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21428
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21429
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21428
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21429
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21429
Committed: http://github.org/Juniper/contrail-controller/commit/f43128cfa797ec5069c9d74bccc4550b7ddbac84
Submitter: Zuul
Branch: master

commit f43128cfa797ec5069c9d74bccc4550b7ddbac84
Author: Nischal Sheth <email address hidden>
Date: Fri Jun 24 08:56:41 2016 -0700

Support large number of routing instances

Following changes are implemented:

- Use bgp::ShowCommand task for neighbor introspect. This makes the
intropsect commands more responsive because they no longer have to
run exclusively with db::DBTable tasks.
- Fix BgpServer::VisitBgpPeers to not iterate all routing instances.
Instead we can simply go through the BgpPeerList in BgpServer.

Change-Id: I6557b5f6fb1d00cddbb5072ca97a331c3f9c30fa
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21447
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21454
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21465
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21466
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21466
Committed: http://github.org/Juniper/contrail-controller/commit/388fbf9ccb2885c7dc281c3aaed403f16b089d29
Submitter: Zuul
Branch: master

commit 388fbf9ccb2885c7dc281c3aaed403f16b089d29
Author: Nischal Sheth <email address hidden>
Date: Mon Jun 27 13:55:22 2016 -0700

Reduce memory footprint for RoutingInstance

Following changes are implemented:

- Do not allocate RouteAggregator if not required
- Do not allocate StaticRouteMgr if not required
- Do not allocate PathResolver if not required

Change-Id: I2f17e8adfb7760485db2857560d0944c3e53e41a
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21476
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21477
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21454
Committed: http://github.org/Juniper/contrail-controller/commit/05afa59f8233ca0363de17232364c8fa5c5fab13
Submitter: Zuul
Branch: master

commit 05afa59f8233ca0363de17232364c8fa5c5fab13
Author: Nischal Sheth <email address hidden>
Date: Sun Jun 26 00:35:22 2016 -0700

Use separate task for IFMap DB

In scaled setups it was observed that running the graph walker stalled
bgp/xmpp peer join/leave processing. This happens because graph walker
runs in db::DBTable task, which is exclusive with bgp::PeerMembership.
Since bgp::PeerMembership cannot run when graph walker is running, peer
join/leave processing gets delayed as well.

Address this by running all the IFMap server side code that used to run
under db::DBTable in a new db::IFMapTable task.

Following changes are implemented:

- Update DB infra to use a user specified task for a given DB, with the
default task as db::DBTable.
- Eliminate use of static members to store task ids so that different DB
instances can use different tasks.
- Fix production and test code to initialize config DB with new task.

Note that agent IFMap code is unchanged.

Change-Id: Id30c89840690e8f41ebf3b277eaedb150c09d123
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21477
Committed: http://github.org/Juniper/contrail-controller/commit/4c72421c3a6f3646fde95b51e347441404590252
Submitter: Zuul
Branch: master

commit 4c72421c3a6f3646fde95b51e347441404590252
Author: Nischal Sheth <email address hidden>
Date: Mon Jun 27 17:21:35 2016 -0700

Use a single partition for ErmVpnTable

There's no need to use multiple partitions for ErmVpnTable till we
support a large number of groups. Existing use cases have exactly
1 multicast group per VN.

Change-Id: If5610ef343a896b7fe53b59dde71b0ec6dc6ea76
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21493
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21504
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21476
Committed: http://github.org/Juniper/contrail-controller/commit/9ebd8d91d010262c4bd5e59bda12164ccc0296d0
Submitter: Zuul
Branch: R3.0

commit 9ebd8d91d010262c4bd5e59bda12164ccc0296d0
Author: Nischal Sheth <email address hidden>
Date: Mon Jun 27 17:21:35 2016 -0700

Use a single partition for ErmVpnTable

There's no need to use multiple partitions for ErmVpnTable till we
support a large number of groups. Existing use cases have exactly
1 multicast group per VN.

Change-Id: If5610ef343a896b7fe53b59dde71b0ec6dc6ea76
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21447
Committed: http://github.org/Juniper/contrail-controller/commit/78e3cdfe2eda73e35abada14a664702ee09a8e2b
Submitter: Zuul
Branch: R3.0

commit 78e3cdfe2eda73e35abada14a664702ee09a8e2b
Author: Nischal Sheth <email address hidden>
Date: Sun Jun 26 00:35:22 2016 -0700

Use separate task for IFMap DB

In scaled setups it was observed that running the graph walker stalled
bgp/xmpp peer join/leave processing. This happens because graph walker
runs in db::DBTable task, which is exclusive with bgp::PeerMembership.
Since bgp::PeerMembership cannot run when graph walker is running, peer
join/leave processing gets delayed as well.

Address this by running all the IFMap server side code that used to run
under db::DBTable in a new db::IFMapTable task.

Following changes are implemented:

- Update DB infra to use a user specified task for a given DB, with the
default task as db::DBTable.
- Eliminate use of static members to store task ids so that different DB
instances can use different tasks.
- Fix production and test code to initialize config DB with new task.

Note that agent IFMap code is unchanged.

Change-Id: Id30c89840690e8f41ebf3b277eaedb150c09d123
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21465
Committed: http://github.org/Juniper/contrail-controller/commit/f711d2bf4a9dbe3f33733d4b3241f8cf7b58c827
Submitter: Zuul
Branch: R3.0

commit f711d2bf4a9dbe3f33733d4b3241f8cf7b58c827
Author: Nischal Sheth <email address hidden>
Date: Mon Jun 27 13:55:22 2016 -0700

Reduce memory footprint for RoutingInstance

Following changes are implemented:

- Do not allocate RouteAggregator if not required
- Do not allocate StaticRouteMgr if not required
- Do not allocate PathResolver if not required

Change-Id: I2f17e8adfb7760485db2857560d0944c3e53e41a
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21428
Committed: http://github.org/Juniper/contrail-controller/commit/436db55a8b5a996b01512d7353eb6d1786f43c5b
Submitter: Zuul
Branch: R3.0

commit 436db55a8b5a996b01512d7353eb6d1786f43c5b
Author: Nischal Sheth <email address hidden>
Date: Fri Jun 24 08:56:41 2016 -0700

Support large number of routing instances

Following changes are implemented:

- Use bgp::ShowCommand task for neighbor introspect. This makes the
intropsect commands more responsive because they no longer have to
run exclusively with db::DBTable tasks.
- Fix BgpServer::VisitBgpPeers to not iterate all routing instances.
Instead we can simply go through the BgpPeerList in BgpServer.

Change-Id: I6557b5f6fb1d00cddbb5072ca97a331c3f9c30fa
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21507
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/21508
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21507
Committed: http://github.org/Juniper/contrail-controller/commit/02a37f56d52391c7cca1821ec9359913b0d9b6ca
Submitter: Zuul
Branch: R3.0

commit 02a37f56d52391c7cca1821ec9359913b0d9b6ca
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 28 15:45:12 2016 -0700

Use db::IFMapTable for IFMap in contrail-dns as well

Change-Id: I640ab43b88b6430bfe7a9aad9bbc6b3e6bddea92
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21493
Committed: http://github.org/Juniper/contrail-controller/commit/1992a4523c7b02eb363acae51669ab7731e7cd0c
Submitter: Zuul
Branch: master

commit 1992a4523c7b02eb363acae51669ab7731e7cd0c
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 28 08:32:46 2016 -0700

Use db::IFMapTable for IFMap in contrail-dns as well

Change-Id: Icd93dacbc2fc49103f6309a2cc72aeabeb7b8378
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21504
Committed: http://github.org/Juniper/contrail-controller/commit/4d3d48ca7e5e06e5eb0f2d716c77dc6e524104af
Submitter: Zuul
Branch: master

commit 4d3d48ca7e5e06e5eb0f2d716c77dc6e524104af
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 28 13:40:45 2016 -0700

Allocate PeerManager only if needed

Change-Id: Ic52a391ee0fcd197a8d2e76c0e5912ca387aafe7
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/21508
Committed: http://github.org/Juniper/contrail-controller/commit/239490e66bcefe4e5f710f2de0497f69f569afe2
Submitter: Zuul
Branch: R3.0

commit 239490e66bcefe4e5f710f2de0497f69f569afe2
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 28 13:40:45 2016 -0700

Allocate PeerManager only if needed

Change-Id: Ic52a391ee0fcd197a8d2e76c0e5912ca387aafe7
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22237
Submitter: Ananth Suryanarayana (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22518
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22518
Committed: http://github.org/Juniper/contrail-controller/commit/89df846de17a5b1b58b73c6d4f2afc246224d289
Submitter: Zuul
Branch: master

commit 89df846de17a5b1b58b73c6d4f2afc246224d289
Author: Nischal Sheth <email address hidden>
Date: Thu Jul 14 09:49:55 2016 -0700

Speed up creation of RoutingInstances by using multiple Tasks

Profiling in scaled scenarios with a large number of VNs showed that we
spend a significant amount of time in creating and/or updating routing
instances.

Instead of creating instances from a single bgp::Config task, use many
bgp::ConfigHelper tasks. The bulk of the processing for instance create
and update operations can happen in parallel.

Use mutexes to serialize access to certain shared resources such as the
following:

o DB
o DBPartition
o BgpConditionListener
o RoutingInstanceMgr
o RoutePathReplicator
o ServiceChainMgr
o RoutingPolicyMgr

Change-Id: Icf83f31ca981e3b8cca98bd26ed5796abe83ede9
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22837
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22837
Committed: http://github.org/Juniper/contrail-controller/commit/732301161589d957ea56ec02a11572debc99a463
Submitter: Zuul
Branch: master

commit 732301161589d957ea56ec02a11572debc99a463
Author: Nischal Sheth <email address hidden>
Date: Tue Aug 2 15:44:46 2016 -0700

Make some minor improvements for routing instance scaling

Following changes are implemented:

- Optimization in RoutePathReplicator to notify RtGroup only if needed
- Allow ifmap::StateMachine and bgp::ConfigHelper to run in parallel
- DBTable::NotifyAllEntries skips table walk if the table is empty

Change-Id: I83a38cab81823a262e4235a78e1c87302a9023ee
Partial-Bug: 1596089

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/22915
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/22915
Committed: http://github.org/Juniper/contrail-controller/commit/6d40fb5444fc6653200078e7567a04984b9e2dd1
Submitter: Zuul
Branch: master

commit 6d40fb5444fc6653200078e7567a04984b9e2dd1
Author: Nischal Sheth <email address hidden>
Date: Thu Aug 4 17:15:03 2016 -0700

Send PollRequest prior to reading PollResponse

Since reading a PollResponse is an expensive operation, send the next
PollRequest to the IFMap server before performing the read. This lets
the IFMap server start building the next PollResponse while we are
processing the previous one.

Note that this means that we do not go back to SendPoll state after
entering PollResponseWait. We stay in PollResponseWait as long we can
successfully send PollRequests and read PollResponses.

This optimization results in a 15% reduction in the time required to
get initial configuration from IFMap server in scaled scenarios.

Change-Id: I5599c897f13ee3a256b2c6b776b4dc69e665af1a
Partial-Bug: 1596089