mainline: CB build 62 snmp-collector and contrail-topology fails with Database:rabbitmq connection down

[Sudheendra Rao]

On mainline CB build62 mitaka, the snmp-collector and contrail-topology fails with error Database:rabbitmq connection down.
root@nodem14:~# docker exec -it analytics contrail-status
== Contrail Analytics ==
contrail-alarm-gen active
contrail-analytics-api active
contrail-analytics-nodemgr active
contrail-collector initializing (Database:nodem14:Global connection down)
contrail-query-engine active
contrail-snmp-collector initializing (Database:RabbitMQ[] connection down)
contrail-topology initializing (Database:RabbitMQ[] connection down)

Additional configs for rabbitMQ connection:-

root@nodem14(analytics):/# grep -rn bbit /etc/contrail/contrail-collector.conf
148:rabbitmq_vhost = /
149:rabbitmq_server_list = 10.10.10.14:5672 10.10.10.6:5672 10.10.10.7:5672
150:rabbitmq_password = A1F5FD330005
151:rabbitmq_user = guest
root@nodem14(analytics):/# grep -rn bbit /etc/contrail/contrail-snmp-collector.conf
32:rabbitmq_vhost = /
33:rabbitmq_server_list = 10.10.10.14:5672 10.10.10.6:5672 10.10.10.7:5672
34:rabbitmq_password = A1F5FD330005
35:rabbitmq_user = guest
root@nodem14(analytics):/# grep -rn bbit /etc/contrail/contrail-topology.conf
30:rabbitmq_vhost = /
31:rabbitmq_server_list = 10.10.10.14:5672 10.10.10.6:5672 10.10.10.7:5672
32:rabbitmq_password = A1F5FD330005
33:rabbitmq_user = guest
root@nodem14(analytics):/#

root@nodem14(analytics):/# tail -n 2 /var/log/contrail/contrail-snmp-collector.log | grep bbit
09/26/2017 03:32:38 PM [contrail-snmp-collector]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = nodem14 process_status = [ << module_id = contrail-snmp-collector instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.10.10.14:2181, 10.10.10.6:2181, 10.10.10.7:2181, ] status = Up description = >>, << type = Collector name = server_addrs = [ 10.10.10.7:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.10.10.14:5672 10.10.10.6:5672 10.10.10.7:5672, ] status = Initializing description = >>, ] description = Collector, Database:RabbitMQ[] connection down >>, ] >>
root@nodem14(analytics):/#

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/35962
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/35963
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/35962
Committed: http://github.com/Juniper/contrail-controller/commit/36499571224fd1887486ad2d18e6e01ae8b5d388
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit 36499571224fd1887486ad2d18e6e01ae8b5d388
Author: Sundaresan Rajangam <email address hidden>
Date: Tue Sep 26 12:25:13 2017 -0700

VncKombuClient to accept rabbit_hosts as comma or space separated
ip:port

Change-Id: I79d7cbc17e7222f4f383c56a9b27f7900942cd55
Closes-Bug: #1719704

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/35963
Committed: http://github.com/Juniper/contrail-controller/commit/9e29db4f7529fd55a2df7cd062d672aead63b25e
Submitter: Zuul (<email address hidden>)
Branch: master

commit 9e29db4f7529fd55a2df7cd062d672aead63b25e
Author: Sundaresan Rajangam <email address hidden>
Date: Tue Sep 26 12:25:13 2017 -0700

VncKombuClient to accept rabbit_hosts as comma or space separated
ip:port

Change-Id: I79d7cbc17e7222f4f383c56a9b27f7900942cd55
Closes-Bug: #1719704