discovery: rebalancing for sandesh clients does not happen

[Raj Reddy]

we need to have some rebalancing done for sandesh clients, otherwise we are ending up with very uneven distribution especially in cases of HA events [restart of collectors]

See attached screenshot where one collector ended up with all remote generators.. [the 7 connected to others are local to the collector]

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15404
Submitter: Deepinder Setia (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15404
Committed: http://github.org/Juniper/contrail-controller/commit/43564524c9ba4642e6ea5b25e56a4ffe7834e999
Submitter: Zuul
Branch: master

commit 43564524c9ba4642e6ea5b25e56a4ffe7834e999
Author: Deepinder Setia <email address hidden>
Date: Tue Nov 24 17:03:19 2015 -0800

Support live load-balancing for specific service type

Auto load-balance is triggered if some servers are more than
5% off expected average allocation. To enable, set load-balance
to True in discovery configuration file. For example:

...
[Collector]
load-balance=True
...

Live load-balancing knob is not global. It must be set per service
as needed.

Change-Id: I6189f1875bedfbdd2b43a85d665fd6ad1e33a2c3
Closes-Bug: #1484293

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/19498
Submitter: Deepinder Setia (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/19498
Committed: http://github.org/Juniper/contrail-controller/commit/90f12c951a53d0cec61aa952881f28b03d57e29e
Submitter: Zuul
Branch: master

commit 90f12c951a53d0cec61aa952881f28b03d57e29e
Author: Deepinder Setia <email address hidden>
Date: Wed Apr 20 16:52:54 2016 -0700

If policy is dynamically load balance, we now pick an overweight assignment
randomly to replace. Previously new publisher would always be sent as the last
assignment. Secondly, load-balance API has two modes - partial and full. With
latter, we wipe out all assignment for specified service type and start
afresh. Partial load-balance behaviour is similar to dynamic load-balance, that
is, we pick one overweight assignment randomly to replace. We also introduce
a new policy based on consistent hashing that implicitly pefroms dynamic load
balancing.

Change-Id: I4ffedae14f7e6c643fda68821a4a2c6bf938b5fd
Fixes-Bug: #1484293

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20548
Submitter: Deepinder Setia (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20548
Committed: http://github.org/Juniper/contrail-controller/commit/e7bb6513f3678b2f13cb36a02a0a4c69a02e7072
Submitter: Zuul
Branch: R3.0

commit e7bb6513f3678b2f13cb36a02a0a4c69a02e7072
Author: Deepinder Setia <email address hidden>
Date: Wed Apr 20 16:52:54 2016 -0700

If policy is dynamically load balance, we now pick an overweight assignment
randomly to replace. Previously new publisher would always be sent as the last
assignment. Secondly, load-balance API has two modes - partial and full. With
latter, we wipe out all assignment for specified service type and start
afresh. Partial load-balance behaviour is similar to dynamic load-balance, that
is, we pick one overweight assignment randomly to replace. We also introduce
a new policy based on consistent hashing that implicitly pefroms dynamic load
balancing.

Change-Id: I4ffedae14f7e6c643fda68821a4a2c6bf938b5fd
Fixes-Bug: #1484293
(cherry picked from commit 90f12c951a53d0cec61aa952881f28b03d57e29e)