Deployment Telemetry plugin with kafka plugin fails

Bug #1673737 reported by Alex Kholkin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel Plugins
Fix Released
Critical
Nadya Privalova

Bug Description

Deployment Telemetry plugin from master with this 2 patches: https://review.openstack.org/#/c/439557
https://review.openstack.org/#/c/446509
and kafka plugin from master fails with next trace from puppet.log:

2017-03-16 20:09:03 ERR (/Stage[main]/Main/Exec[run_/tmp/create-topics.sh]/returns) change from notrun to 0 failed: /tmp/create-topics.sh returned 1 instead of one of [0]

Steps to reproduce:
1. Deploy Mos 10.0
2. Install Telemetry Plugin with Kafka Plugin
3. Add 3 controllers with kafka and 2 computes with cinder
4. Enable Fuel Telemetry Plugin. Use Advanced settings, Ceilometer Event API, Ceilometer Resource API. Use external InfluxDB and external Elasticsearch. Enable Kafka Plugin.
5. Deploy env

Expected result:
Env is successfully deployed

Actual result:
Deployment fails

Additional information:

root@node-2:~# /opt/kafka/bin/kafka-topics.sh --create --if-not-exists --zookeeper 10.109.1.2:2181 --replication-factor 3 --partitions 2 --topic notifications.sample
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Error while executing topic command : replication factor: 3 larger than available brokers: 1
[2017-03-17 09:36:39,945] ERROR kafka.admin.AdminOperationException: replication factor: 3 larger than available brokers: 1
 at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:117)
 at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:403)
 at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:110)
 at kafka.admin.TopicCommand$.main(TopicCommand.scala:61)
 at kafka.admin.TopicCommand.main(TopicCommand.scala)
 (kafka.admin.TopicCommand$)

Revision history for this message
Alex Kholkin (akholkin) wrote :

status of the controller where kafka is down:

● kafka.service - Apache Kafka server (broker)
   Loaded: loaded (/lib/systemd/system/kafka.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2017-03-17 10:40:11 UTC; 2min 55s ago
     Docs: http://kafka.apache.org/documentation.html
  Process: 8425 ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties (code=exited, status=1/FAILURE)
 Main PID: 8425 (code=exited, status=1/FAILURE)

trace from kafka/server.log:

[2017-03-17 10:40:11,574] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
        at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)
        at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75)
        at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57)
        at kafka.server.KafkaServer.initZk(KafkaServer.scala:294)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:180)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
        at kafka.Kafka$.main(Kafka.scala:67)
        at kafka.Kafka.main(Kafka.scala)
[2017-03-17 10:40:11,576] INFO shutting down (kafka.server.KafkaServer)

Changed in fuel-plugins:
status: New → Incomplete
status: Incomplete → Confirmed
Changed in fuel-plugins:
assignee: nobody → Nadya Privalova (nprivalova)
Revision history for this message
Alex Kholkin (akholkin) wrote :
Changed in fuel-plugins:
status: Confirmed → In Progress
Alex Kholkin (akholkin)
Changed in fuel-plugins:
status: In Progress → Fix Committed
Changed in fuel-plugins:
importance: Undecided → Critical
Alex Kholkin (akholkin)
Changed in fuel-plugins:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.