R.3.1 build 33 : zookeeper ip not correct in analytics conf files

[Ankit Jain]

Most of the scripts in our sanity failed due to this issue.

Zookeeper seems to have moved to config node from db node through the below commit:
https://bugs.launchpad.net/juniperopenstack/+bug/1619803

Analytics conf files (contrail-alarm-gen.conf, contrail-topology.conf and contrail-snmp-collector.conf) are not updated as per this change made. Alalytics services still trying to connect to zookeeper believing it to be on db node.

In contrail-alarm-gen.conf, contrail-topology.conf and contrail-snmp-collector.conf:

zk_server_ip=10.204.217.179:2181 <- it should have been 10.204.217.172:2181 as zookeeper running on nodei13-vm2 not nodei14-vm2

Config Nodes : [u'nodei13-vm2']
Control Nodes : [u'nodei13-vm2', u'nodei14-vm2']
Compute Nodes : [u'nodea12', u'nodea15', u'nodea17']
Openstack Node : [u'nodek7-vm2']
WebUI Node : nodei13-vm2
Analytics Nodes : [u'nodei13-vm2']

root@nodei13-vm2:~# contrail-status
== Contrail Control ==
supervisor-control: active
contrail-control active
contrail-control-nodemgr active
contrail-dns active
contrail-named active

== Contrail Analytics ==
supervisor-analytics: active
contrail-alarm-gen initializing (Zookeeper:Zookeeper[Connection time-out] connection down)
contrail-analytics-api initializing (UvePartitions:UVE-Aggregation[Partitions:0] connection down)
contrail-analytics-nodemgr active
contrail-collector active
contrail-query-engine active
contrail-snmp-collector initializing (Zookeeper:Zookeeper[Connection time-out] connection down)
contrail-topology initializing (Zookeeper:Zookeeper[Connection time-out] connection down)

== Contrail Config ==
supervisor-config: active
contrail-api:0 active
contrail-config-nodemgr active
contrail-device-manager active
contrail-discovery:0 active
contrail-schema active
contrail-svc-monitor active
ifmap active

== Contrail Web UI ==
supervisor-webui: active
contrail-webui active
contrail-webui-middleware active

== Contrail Support Services ==
supervisor-support-service: inactive
unix:///var/run/supervisord_support_service.sockno

[Ignatious Johnson Christopher (ijohnson-x)]

By any chance, did you use old fab to provision this setup?

[Ankit Jain (ankitja)]

No, this was sanity setup. sanity setup picks the latest fab.

[Sudheendra Rao (sudheendra-k)]

Ignatious,

    This is contrail-networking pkg sanity setup. testbed.py FYI:
https://github.com/Juniper/contrail-tools/blob/master/testbeds/testbed_virtual_a12_a15_a17.py.networking.ubuntu-14.04

[Ignatious Johnson Christopher (ijohnson-x)]

Looking at the failed sanity logs http://anamika.englab.juniper.net:8080/job/Contrail_Networking_ubuntu-14-04_kilo_Multi_Node_Sanity/79/consoleFull

I see that incorrect ip is set in contrail-snmp-collector.conf and contrail-topology.conf.

Logs:
------
2016-09-14 12:29:01:645993: [root@10.204.217.172] out: [localhost] local: contrail-config --set /etc/contrail/contrail-snmp-collector.conf DEFAULTS zookeeper '10.204.217.179:2181'

2016-09-14 12:29:01:789577: [root@10.204.217.172] out: [localhost] local: contrail-config --set /etc/contrail/contrail-topology.conf DEFAULTS zookeeper '10.204.217.179:2181'

However bug description reports contrail-analytics-api.conf is set with incorrect zk_server_ip, From the code(https://github.com/Juniper/contrail-provisioning/blob/R3.1/contrail_provisioning/collector/setup.py#L279) this should be set with correct ip, as we are passing correct ip's in the command line.

Logs:
------
2016-09-14 12:28:59:376594: setup-vnc-collector --cassandra_ip_list 10.204.217.179 --zookeeper_ip_list 10.204.217.172 --amqp_ip_list 10.204.216.111 --amqp_port 5672 --cfgm_ip 10.204.217.172 --self_collector_ip 10.204.217.172 --num_nodes 1 --analytics_syslog_port 19876 --kafka_enabled True --keystone_ip 10.204.216.111 --keystone_version v2.0 --keystone_admin_user admin --keystone_admin_passwd contrail123 --keystone_admin_tenant_name admin --keystone_auth_protocol http --keystone_auth_port 35357 --keystone_insecure True --aaa_mode cloud-admin

Please confirm the config file that are populated with incorrect zookeeper ip.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24223
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24224
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24225
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[Ankit Jain (ankitja)]

Hi Ignatious,

You are right. Files polulated with incorrect ips are contrail-alarm-gen.conf, contrail-topology.conf and contrail-snmp-collector.conf...I corrected the bug description now.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24225
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24223
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24224
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24225
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/24223
Submitter: Ignatious Johnson Christopher (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24223
Committed: http://github.org/Juniper/contrail-provisioning/commit/2745a26e6bd0814e63c5c16f368c80d916a31d75
Submitter: Zuul
Branch: master

commit 2745a26e6bd0814e63c5c16f368c80d916a31d75
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Sep 16 11:53:10 2016 -0700

Issue
cassandra ip is set as zookeeper ip's and it working earlier as the
cassanda and zookeeper was running in same node, now after moving zookeeper
to config node, we need ot correct it.

Fix
Setting zookeeper ip's recived from commandline is used to set the zookeeper
parameter in contrail-snmp-collector.conf and contrail-topology.conf files.

Change-Id: If58b89cebfb4941b4e481972f83f657f3a3d0c54
Closes-Bug: 1623791

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24224
Committed: http://github.org/Juniper/contrail-provisioning/commit/3d10eb9ad52069948e35aec779fe173dc3f5e3ee
Submitter: Zuul
Branch: R3.1

commit 3d10eb9ad52069948e35aec779fe173dc3f5e3ee
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Sep 16 11:53:10 2016 -0700

Issue:
cassandra ip is set as zookeeper ip's and it working earlier as the
cassanda and zookeeper was running in same node, now after moving zookeeper
to config node, we need ot correct it.

Fix:
Setting zookeeper ip's recived from commandline is used to set the zookeeper
parameter in contrail-snmp-collector.conf and contrail-topology.conf files.

Change-Id: If58b89cebfb4941b4e481972f83f657f3a3d0c54
Closes-Bug: 1623791

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/24225
Committed: http://github.org/Juniper/contrail-provisioning/commit/060df0121a05f0c05d45916603dbb0286182a979
Submitter: Zuul
Branch: R3.0

commit 060df0121a05f0c05d45916603dbb0286182a979
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Sep 16 11:53:10 2016 -0700

Issue:
cassandra ip is set as zookeeper ip's and it was working earlier as the
cassanda and zookeeper was running in same node, now after moving zookeeper
to config node, need to correct it and point to config node.

Fix:
Setting zookeeper ip's recived from commandline is used to set the zookeeper
parameter in contrail-snmp-collector.conf and contrail-topology.conf files.

Change-Id: If58b89cebfb4941b4e481972f83f657f3a3d0c54
Closes-Bug: 1623791