In a multi-controller setup, kube-manager is not connecting to contrail-api

[Vedamurthy Joshi]

R4.0 Build 7 Ubuntu 16.04.2 container controller setup

On a multi-contrail-controller setup, i have 3 controller.
In contrail-kubernetes.conf, we had this
vnc_endpoint_ip = 10.204.216.58,10.204.216.59,10.204.216.60
With such a config, kube-manager does not connect to contrail-api at all.

If it is just one IP, i.e
vnc_endpoint_ip = 10.204.216.58
things are fine.

(Pdb) n
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(131)_vnc_connect()
-> try:
(Pdb) l
126 # Retry till API server connection is up
127 connected = False
128 import pdb; pdb.set_trace()
129 self.connection_state_update(ConnectionStatus.INIT)
130 while not connected:
131 -> try:
132 vnc_lib = VncApi(self.args.auth_user,
133 self.args.auth_password, self.args.auth_tenant,
134 self.args.vnc_endpoint_ip, self.args.vnc_endpoint_port,
135 auth_token_url=self.args.auth_token_url)
136 connected = True
(Pdb) n
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(132)_vnc_connect()
-> vnc_lib = VncApi(self.args.auth_user,
(Pdb) l
127 connected = False
128 import pdb; pdb.set_trace()
129 self.connection_state_update(ConnectionStatus.INIT)
130 while not connected:
131 try:
132 -> vnc_lib = VncApi(self.args.auth_user,
133 self.args.auth_password, self.args.auth_tenant,
134 self.args.vnc_endpoint_ip, self.args.vnc_endpoint_port,
135 auth_token_url=self.args.auth_token_url)
136 connected = True
137 self.connection_state_update(ConnectionStatus.UP)
(Pdb) n
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(133)_vnc_connect()
-> self.args.auth_password, self.args.auth_tenant,
(Pdb) n
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(134)_vnc_connect()
-> self.args.vnc_endpoint_ip, self.args.vnc_endpoint_port,
(Pdb)
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(135)_vnc_connect()
-> auth_token_url=self.args.auth_token_url)
(Pdb) n
ConnectionError: ConnectionError()
> /usr/lib/python2.7/dist-packages/kube_manager/vnc/vnc_kubernetes.py(135)_vnc_connect()
-> auth_token_url=self.args.auth_token_url)
(Pdb) l
130 while not connected:
131 try:
132 vnc_lib = VncApi(self.args.auth_user,
133 self.args.auth_password, self.args.auth_tenant,
134 self.args.vnc_endpoint_ip, self.args.vnc_endpoint_port,
135 -> auth_token_url=self.args.auth_token_url)
136 connected = True
137 self.connection_state_update(ConnectionStatus.UP)
138 except requests.exceptions.ConnectionError as e:
139 # Update connection info
140 self.connection_state_update(ConnectionStatus.DOWN, str(e))
(Pdb) print self.args.vnc_endpoint_ip
10.204.216.58,10.204.216.59,10.204.216.60
(Pdb) l
141 time.sleep(3)
142 except ResourceExhaustionError:
143 time.sleep(3)
144 return vnc_lib
145
146 def _sync_km(self):
147 for cls in DBBaseKM.get_obj_type_map().values():
148 for obj in cls.list_obj():
149 cls.locate(obj['uuid'], obj)
150
151 @staticmethod
(Pdb) n

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/31910
Submitter: Sachchidanand Vaidya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/31910
Committed: http://github.com/Juniper/contrail-controller/commit/1961ec9ec28096f0721136d45430983f90a2ce1e
Submitter: Zuul (<email address hidden>)
Branch: R4.0

commit 1961ec9ec28096f0721136d45430983f90a2ce1e
Author: Sachchidanand Vaidya <email address hidden>
Date: Fri May 19 17:18:25 2017 -0700

- Pass multiple api servers as list to VncApi()

Change-Id: Icc8b1d6bfc0ed3f47ad10e3c353dabc5b42dbeb3
Closes-bug: #1691997

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32106
Submitter: Sachchidanand Vaidya (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/32106
Committed: http://github.com/Juniper/contrail-controller/commit/dac47222c0846687c0ba5f74e765a8c972dafce5
Submitter: Zuul (<email address hidden>)
Branch: master

commit dac47222c0846687c0ba5f74e765a8c972dafce5
Author: Sachchidanand Vaidya <email address hidden>
Date: Fri May 19 17:18:25 2017 -0700

- Pass multiple api servers as list to VncApi()

Change-Id: Icc8b1d6bfc0ed3f47ad10e3c353dabc5b42dbeb3
Closes-bug: #1691997