Tracking bug for BGPaaS implementation

[Nischal Sheth]

Tracking bug for BGPaaS implementation.
Includes schema changes, ST/SVCMON changes, CN configuration changes.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15369
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15370
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15369
Committed: http://github.org/Juniper/contrail-controller/commit/87a0293ea94bc1eeaabefbe97ff76e51a7865c8b
Submitter: Zuul
Branch: master

commit 87a0293ea94bc1eeaabefbe97ff76e51a7865c8b
Author: Nischal Sheth <email address hidden>
Date: Thu Nov 19 16:07:57 2015 -0800

Treat bgp-router for BGPaaS as local

This is temporary till we use the router-type field to identify
bgp-routers that are created to represent BGPaaS servers. Schema
changes for that will be made soon.

Highlights:

- Each CN treats BGPaaS server as being equivalent to self
- Master instance ASN and identifier are used for such bgp neighbors
- Neighbors in non-master instances are automatically set to passive
- Add unit tests to exercise new code

Change-Id: Id40176e06c4a95e0271f008f9612edfba7b1c2b1
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15370
Committed: http://github.org/Juniper/contrail-controller/commit/355b88abae1cc27f2f4c247ea929177b790464d6
Submitter: Zuul
Branch: master

commit 355b88abae1cc27f2f4c247ea929177b790464d6
Author: Nischal Sheth <email address hidden>
Date: Fri Nov 20 18:37:12 2015 -0800

Extend introspect for config peering/neighbors to all instances

Since BGPaaS configures peerings and neighbors in non-master instances,
extend introspect for config peerings and neighbors to show objects in
all instances.

Highlights:

- Fix syntax error in ShowBgpNeighborFamily sandesh definition
- Move code for neighbors to bgp_show_config.cc
- Move code for peerings to bgp_show_ifmap_peering_config.cc
- Add level of indirection to avoid linking issues for peering show
command. This is needed because YAML config does not have notion of
peering.
- Paginate output for peerings and neighbors while we're at it
- Tweak tests so that they pass with the new code

Change-Id: Ic5dc47a7684e80ac693aea1e2660abff783ce681
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15403
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15403
Committed: http://github.org/Juniper/contrail-controller/commit/ad4d79773bbeebc0bac6db573fab847408e9946c
Submitter: Zuul
Branch: master

commit ad4d79773bbeebc0bac6db573fab847408e9946c
Author: Sachin Bansal <email address hidden>
Date: Tue Nov 24 16:55:53 2015 -0800

Schema changes for BGPaaS

Change-Id: Idff3c8dd454a3a3393419751b367bac42c3b29dc
Related-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15432
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15432
Committed: http://github.org/Juniper/contrail-controller/commit/75d4b3d0dd8419a5a53be583281423326b5c8667
Submitter: Zuul
Branch: master

commit 75d4b3d0dd8419a5a53be583281423326b5c8667
Author: Sachin Bansal <email address hidden>
Date: Wed Nov 25 11:37:43 2015 -0800

More schema changes for BGPaaS

Change-Id: I48465a3bd2d4cb7a9739510789d8e6c247b76bdc
Related-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15506
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15506
Committed: http://github.org/Juniper/contrail-controller/commit/53ef8480980ff24f90b337b1498313d8f95d9842
Submitter: Zuul
Branch: master

commit 53ef8480980ff24f90b337b1498313d8f95d9842
Author: Nischal Sheth <email address hidden>
Date: Wed Nov 25 09:34:07 2015 -0800

Maintain endpoint to BgpPeer map in BgpServer

This map is used to lookup a BGPaaS BgpPeer from the source address and
source port of an incoming TCP session. If an exact match is not found,
the source address is used to find matching BgpPeer in master instance.

Highlights:

- Move relevant schema snippet from vnc_cfg.xsd to bgp_schema.xsd
- Add a remote_endpoint field to BgpNeighborConfig
- Populate new field from bgp-as-a-service_parameters property of client
- Update ifmap configuration code to use the router-type instead of name
- Set and update router_type and remote_endpoint fields in BgpPeer
- Resolve paths learnt from BgpPeers of router_type bgpaas-client
- Tweak introspect to show BgpPeers from non-master instances as well
- Add new tests to bgp_ifmap_config_manager_test and bgp_config_test

Change-Id: I93aeeba0d48942a463b772f9503932b50a640764
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15553
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15561
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15561
Committed: http://github.org/Juniper/contrail-controller/commit/40de43080f177b88c0bd6defd1da0eaf86135877
Submitter: Zuul
Branch: master

commit 40de43080f177b88c0bd6defd1da0eaf86135877
Author: Nischal Sheth <email address hidden>
Date: Tue Dec 1 11:14:23 2015 -0800

Further progress on BGPaaS implementation

Highlights:

- Schema updates for BGPaaS
- Do not use BgpAsAServiceParameters as it's being removed
- Use the port number from the remote bgp-router
- Update unit tests based on above changes
- Do not export routes to BgpPeers in non-master instances
- Add router type and port fields to ShowBgpNeighborConfig
- Add router type, peer port and transport address to BgpNeighborResp
- Cosmetic changes to BgpNeighborResp related methods
- Add couple of paranoid checks to RibOut and PeerRibMembershipManager

Change-Id: Id9c06579549f648996e22bf07bd4f79fd47768fa
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15584
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15584
Committed: http://github.org/Juniper/contrail-controller/commit/4a6bdc571c1bdf2b1ca3dc769db54a29700a7c2a
Submitter: Zuul
Branch: master

commit 4a6bdc571c1bdf2b1ca3dc769db54a29700a7c2a
Author: Nischal Sheth <email address hidden>
Date: Thu Dec 3 10:05:48 2015 -0800

Add a source port field to BgpRouterParams

Use an explicit field for source port instead of reusing port field
which is really meant for destination port.

Change-Id: I792bede998c995eebea4d144899e5c7b6431911c
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15622
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15622
Committed: http://github.org/Juniper/contrail-controller/commit/5987bd298cc39073364ff168f469dc4c396c2624
Submitter: Zuul
Branch: master

commit 5987bd298cc39073364ff168f469dc4c396c2624
Author: Sachin Bansal <email address hidden>
Date: Fri Dec 4 16:51:30 2015 -0800

Schema transformer changes for Bgp as a service

Added schema transformer changes and unit test cases for BGP as a service

Change-Id: Ia2739bdfcde9e161179463952d0f2cbec6653dca
Partial-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15659
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15659
Committed: http://github.org/Juniper/contrail-controller/commit/b58699ae4e2af5c8583a4220073d44b23e595b45
Submitter: Zuul
Branch: master

commit b58699ae4e2af5c8583a4220073d44b23e595b45
Author: Sachin Bansal <email address hidden>
Date: Mon Dec 7 15:01:31 2015 -0800

Newly added arguments should be parsed as integers

Also fixed reaction map
Change-Id: I0411d442657270d5429b3efcdc1f13cf60836db2
Related-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15862
Submitter: Sachin Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15862
Committed: http://github.org/Juniper/contrail-controller/commit/5439d6fcd8cfd8bc27de85b5a606dd3f37e42ee0
Submitter: Zuul
Branch: master

commit 5439d6fcd8cfd8bc27de85b5a606dd3f37e42ee0
Author: Sachin Bansal <email address hidden>
Date: Wed Dec 16 13:54:50 2015 -0800

Added schema element for project-bgpaas link

Change-Id: I46450d8367e14798cf0b3d0acc51aa74a5713d30
Related-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15925
Submitter: Manish Singh (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15925
Committed: http://github.org/Juniper/contrail-controller/commit/b515dbf5beb6f04bf6111533cc879c318d0a80e1
Submitter: Zuul
Branch: master

commit b515dbf5beb6f04bf6111533cc879c318d0a80e1
Author: Manish Singh <email address hidden>
Date: Fri Jan 8 14:14:00 2016 +0530

BGP as a service agent changes.

Function:
When a VM tries to establish BGP session it will connect o its gateway or DNS
ip. For example in subnet of 1.1.1.0/24, VM will try to connect on well defined
BGP port to either 1.1.1.1(=gw) or 1.1.1.2(=DNS). Agent sees this traffic and
creates a NAT. The calculation of NAT is done as follows:
Pkt from VM:
source: VM-SIP, destination: DIP(gw/dns), source port: VM-sport, destination
port: BGP-port.

After NAT:
source: vrouter IP, destination: Control-node#1(if DIP was gw),
Control-node#2(if DIP was DNS), source port: BGP-router port,
destination port: BGP-port.

Config object(bgp-router) will provide BGP-router port used in NAT.
If new set of control-node changes flows should use new set given.

In this feature agent implements following:
- Add dependency between VM interface, BGP as service.
- BGP as a service node resides under VmInterface. Using adjacencies it travels
to bgp-as-a-service to extract IP for SNAT and bgp-router (adjacent
to for source port nat.
- Flow - Keeping track of bgp-as-a-service object in VMI to populate
flow.
For destination it uses control-node for destination IP.
- Control-node - rebake all Vm interfaces to reflect changes.
- ifmap - white-listing of nodes bgp-as-a-service, vmi, bgp-router.
- Reserves a set of BGP port which can potentially be used in
bgp-router object.
This is provided via contrail-vrouter-agent.conf and agents binds on
to these ports, so that host does not use it.

Pending items:
- UT.

Closes-Bug: 1518047

Conflicts:
 src/ifmap/ifmap_graph_walker.cc
 src/vnsw/agent/oper/agent.sandesh
 src/vnsw/agent/pkt/pkt_flow_info.h

Change-Id: I12c8d019b4b6805cc5517ad595f4d2c8e9956100

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16226
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16242
Submitter: Nischal Sheth (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16226
Committed: http://github.org/Juniper/contrail-controller/commit/c3dbafd2fe2f6cb8c9604ed8d43f978aee0f1525
Submitter: Zuul
Branch: master

commit c3dbafd2fe2f6cb8c9604ed8d43f978aee0f1525
Author: Nischal Sheth <email address hidden>
Date: Mon Jan 11 16:19:47 2016 -0800

Fix issue with initialization of StateMachine::hold_time_

StateMachine::hold_time_ is initialized from the BgpPeer's configured
hold time. However the BgpPeer's hold_time_ is yet not initialzed when
StateMachine is constructed.

Change-Id: I63b6b683b7c886f08ffbb80da7d25bc46996026b
Closes-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16242
Committed: http://github.org/Juniper/contrail-controller/commit/885b5e5b3ad94c4b7ad35ea5c684c11135916bd2
Submitter: Zuul
Branch: master

commit 885b5e5b3ad94c4b7ad35ea5c684c11135916bd2
Author: Nischal Sheth <email address hidden>
Date: Tue Jan 12 09:49:31 2016 -0800

Add instance name to bgp neighbor introspect output

Change-Id: I71127171e7d7920bc8d36f90037d0da52bf7590d
Closes-Bug: 1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/25673
Submitter: Ananth Suryanarayana (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/25673
Committed: http://github.org/Juniper/contrail-controller/commit/a459ebcac042357afdbdc13c1780dbc3d258361c
Submitter: Zuul
Branch: master

commit a459ebcac042357afdbdc13c1780dbc3d258361c
Author: Ananth Suryanarayana <email address hidden>
Date: Wed Nov 2 11:51:48 2016 -0700

Add integration test for BGPaaS functionality

o Apply BGPaaS server and BGPaaS client configurations to DUT
o Use other BgpServer to peer as BGPaaS clients
o Play with source-port and cheat production code to infer connections coming
as if from BGPaaS client
o Verify that peers come up and stay established
o Verify that cleanup happens correctly
o TODO: Add more integration tests involving mock agents, with BGPaaS routes
that resolve over routes learned from mock XMPP agents

Change-Id: Iaeddd1658f187dd735585bc094b233bcd940dd57
Partial-Bug: #1518047

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/26014
Submitter: Ananth Suryanarayana (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/26014
Committed: http://github.org/Juniper/contrail-controller/commit/b7506d3016983630eed1741d68fd8919d0d234ec
Submitter: Zuul
Branch: master

commit b7506d3016983630eed1741d68fd8919d0d234ec
Author: Ananth Suryanarayana <email address hidden>
Date: Fri Nov 11 14:17:53 2016 -0800

Add routes check to BGPaaS test

Send routes from agent and verify that agent receive BGPaaS route only when
the BGPaaS routes' nexthop is resolvable but not otherwise.

Add the tests for IPv4 and IPv6 routes and also cover ecmp cases

Partial-Bug: #1518047
Change-Id: I888bd401f854974ff1db804b9aab829f5b57bdc0

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/26781
Submitter: Ananth Suryanarayana (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/26781
Committed: http://github.org/Juniper/contrail-controller/commit/f1f8e88e3bb53f5750ab170a15c5d5d9d951593e
Submitter: Zuul (<email address hidden>)
Branch: master

commit f1f8e88e3bb53f5750ab170a15c5d5d9d951593e
Author: Ananth Suryanarayana <email address hidden>
Date: Sun Dec 4 15:22:33 2016 -0800

Stablize bgp_bgpaas_test

Use TASK_UTIL_EXPECT_* over a route lookup and nexthop verfication together
so that updated route if any is used for verification

Change-Id: Iedf176ca2ffbcb2b70555c600033b0d466ae7ba3
Closes-Bug: #1518047