Do not connect to same IFMap server after connection goes down

[Nischal Sheth]

Existing implementation attempts to reconnect to the same IFMap server 2
times in the event of connection close/failure. If it fails to reconnect,
it uses the other IFMap server obtained from discovery.

If the IFMap Server is restarted or crashes and comes back up quickly, CN
may succeed in connecting to it during the 2 attempts. The IFMap Server
will likely not have received the entire config database from API server
at that point. Hence CN will receive an incomplete config database and
perform an audit, thus deleting valid config objects temporarily. This may
cause the CN to stop publishing itself to discovery and/or cause vRouter
agents to get unnecessary deletes for config objects, thus causing traffic
disruption for tenants.

Note that the above behavior happens because IFMap Server doesn't have the
ability to refuse incoming connections from CN till it's config database
is fully populated. API server sets IFMap Server to oper down in discovery
but that doesn't help in above scenario since CN doesn't consult discovery
before reconnecting to same IFMap Server.

Proposed fix is to not try to reconnect to same IFMap Server. This should
fix the problem as long as the other IFMap Server has the complete config
database. This should be the case if multiple IFMap Servers are not being
restarted at the same time or one after the other.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20133
Submitter: Tapan Karwa (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20134
Submitter: Tapan Karwa (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20135
Submitter: Tapan Karwa (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20136
Submitter: Tapan Karwa (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20137
Submitter: Tapan Karwa (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20137
Committed: http://github.org/Juniper/contrail-controller/commit/4b68d7821089b48824ddd3bbdd98fd5e27be8140
Submitter: Zuul
Branch: master

commit 4b68d7821089b48824ddd3bbdd98fd5e27be8140
Author: Tapan Karwa <email address hidden>
Date: Fri May 6 09:42:14 2016 -0700

Do not connect to same IFMap server (irond) after connection flap

Closes-Bug: #1580685

Change-Id: Ie5b4a0a2212420ffdba138702b9b5c06d2e7de18
(cherry picked from commit 8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20136
Committed: http://github.org/Juniper/contrail-controller/commit/8332cfaf2487c863936e5636931ee2b4d6e48186
Submitter: Zuul
Branch: R3.0

commit 8332cfaf2487c863936e5636931ee2b4d6e48186
Author: Tapan Karwa <email address hidden>
Date: Fri May 6 09:42:14 2016 -0700

Do not connect to same IFMap server (irond) after connection flap

Closes-Bug: #1580685

Change-Id: Ie5b4a0a2212420ffdba138702b9b5c06d2e7de18
(cherry picked from commit 8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20135
Committed: http://github.org/Juniper/contrail-controller/commit/15d86582d9da67a4f33405d1a7d15e47c291b7f0
Submitter: Zuul
Branch: R2.22.x

commit 15d86582d9da67a4f33405d1a7d15e47c291b7f0
Author: Tapan Karwa <email address hidden>
Date: Fri May 6 09:42:14 2016 -0700

Do not connect to same IFMap server (irond) after connection flap

Closes-Bug: #1580685

Change-Id: Ie5b4a0a2212420ffdba138702b9b5c06d2e7de18
(cherry picked from commit 8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20134
Committed: http://github.org/Juniper/contrail-controller/commit/28197fe595db52f7d07fde7795ebd6d8573c3d96
Submitter: Zuul
Branch: R2.20

commit 28197fe595db52f7d07fde7795ebd6d8573c3d96
Author: Tapan Karwa <email address hidden>
Date: Fri May 6 09:42:14 2016 -0700

Do not connect to same IFMap server (irond) after connection flap

Closes-Bug: #1580685

Change-Id: Ie5b4a0a2212420ffdba138702b9b5c06d2e7de18
(cherry picked from commit 8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20133
Committed: http://github.org/Juniper/contrail-controller/commit/8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442
Submitter: Zuul
Branch: R2.21.x

commit 8cd9b651e4a9d4e5ed6f7f3ccf3849bf57cad442
Author: Tapan Karwa <email address hidden>
Date: Fri May 6 09:42:14 2016 -0700

Do not connect to same IFMap server (irond) after connection flap

Closes-Bug: #1580685

Change-Id: Ie5b4a0a2212420ffdba138702b9b5c06d2e7de18