[HDP] Cluster isn't configuring using hdp plugin when cluster/templates contains upper letters in name

[Sergey Galkin]

Step to reproduce:
1. Create node group template with name "HDP-master" and Node Processes: NAMENODE, SECONDARY_NAMENODE, AMBARI_SERVER, JOBTRACKER, GANGLIA_SERVER, NAGIOS_SERVER
2. Create node group template with name "hdp-worker" and Node Processes: DATANODE, HDFS_CLIENT, TASKTRACKER, MAPREDUCE_CLIENT
3. Create cluster template with name "hdp-templ" 1 node "HDP-master" and 2 nodes "hdp-worker"
4. Start cluster

Configuration cluster failed with errors :
2013-12-23 15:42:21.737 23207 INFO savanna.service.instances [-] Cluster 'db0ba3aa-750e-4ee7-add7-7f8138e70e04': all instances are accessible
2013-12-23 15:42:21.844 23207 INFO savanna.service.instances [-] Cluster status has been changed: id=db0ba3aa-750e-4ee7-add7-7f8138e70e04, New status=Preparing

2013-12-23 15:42:23.210 23207 INFO savanna.service.api [-] Cluster status has been changed: id=db0ba3aa-750e-4ee7-add7-7f8138e70e04, New status=Configuring
2013-12-23 15:42:23.239 23207 INFO savanna.swift.swift_helper [-] Swift would be integrated with the following params: [{'name': u'fs.swift.service.savanna.auth.url', 'value': 'http://192.168.0.2:35357/v2.0/tokens/', 'description': ''}, {'name': u'fs.swift.service.savanna.tenant', 'value': u'admin', 'description': ''}, {'name': u'fs.swift.service.savanna.http.port', 'value': '8080', 'description': ''}, {'name': u'fs.swift.service.savanna.https.port', 'value': '443', 'description': ''}, {'name': u'fs.swift.service.savanna.public', 'value': 'true', 'description': ''}, {'name': u'fs.swift.service.savanna.auth.endpoint.prefix', 'value': '/endpoints/AUTH_', 'description': ''}, {'name': u'fs.swift.impl', 'value': 'org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem', 'description': ''}]
2013-12-23 15:42:23.239 23207 ERROR savanna.service.api [-] Can't configure cluster 'test' (reason: u'HDP-master')
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api Traceback (most recent call last):
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api File "/usr/lib/python2.6/site-packages/savanna/service/api.py", line 159, in _provision_cluster
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api plugin.configure_cluster(cluster)
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api File "/usr/lib/python2.6/site-packages/savanna/plugins/hdp/ambariplugin.py", line 220, in configure_cluster
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api self.create_cluster(cluster)
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api File "/usr/lib/python2.6/site-packages/savanna/plugins/hdp/ambariplugin.py", line 52, in create_cluster
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api h.HadoopServer(host, cluster_spec.node_groups[host_role],
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api KeyError: u'HDP-master'
2013-12-23 15:42:23.239 23207 TRACE savanna.service.api
2013-12-23 15:42:23.602 23207 INFO savanna.service.api [-] Cluster status has been changed: id=db0ba3aa-750e-4ee7-add7-7f8138e70e04, New status=Configuring

[Sergey Lukjanov (slukjanov)]

@John, please, take a look on it.

[Ting (tul75123)]

Hi, I have this problem too. Does anyone know the solution? thanks

[Dmitry Mescheryakov (dmitrymex)]

Ting,

As far as I recall, the problem was in upper case letters. Use only lower case letters in cluster/templates names and the plugin should work.

[Andrew Lazarev (alazarev)]

Jonh, please confirm that this is a dupe of https://bugs.launchpad.net/savanna/+bug/1232075 and update the bug accordingly.

[Ting (tul75123)]

Dmitry,
Thanks for the solution. After I change all parameters to lower case letter, it works!

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/185355
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=b1b9af117993c0d9dd27476ddbd70cb0c5d22620
Submitter: Jenkins
Branch: master

commit b1b9af117993c0d9dd27476ddbd70cb0c5d22620
Author: Vitaly Gridnev <email address hidden>
Date: Mon May 25 13:38:25 2015 +0300

    Fix issue with configuring HDP cluster

    When node groups of cluster have uppercase letters in name,
    cluster can't be configured. This patch resolve this issue, because
    there is no sense in making node group name lowercase during creating
    Clusterspec object.

    Change-Id: I6ada7468bc1a6826727d330730cdc57c789f70b7
    Closes-bug: 1263722