Sahara service fails to start on TripleO because config file contains hdp plugin

[Sagi (Sergey) Shnaidman]

HDP plugin was deprecated in Sahara:
https://github.com/openstack/sahara/commit/037ba02ea7588d2eae5bf3a82420c5d3868141e7

Config file in tripleo deployment still contains it and sahara services fail to start:
Aug 08 20:12:23 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:23.329 9653 INFO sahara.utils.rpc [-] Notifications disabled
Aug 08 20:12:23 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:23.337 9653 WARNING oslo_config.cfg [-] Option "rpc_backend" from group "DEFAULT" is deprecated for removal. Its value may be silently i
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.447 9653 INFO sahara.plugins.base [-] Plugin vanilla loaded sahara.plugins.vanilla.plugin:VanillaProvider
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.448 9653 INFO sahara.plugins.base [-] Plugin cdh loaded sahara.plugins.cdh.plugin:CDHPluginProvider
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.462 9653 INFO sahara.plugins.base [-] Plugin mapr loaded sahara.plugins.mapr.plugin:MapRPlugin
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.462 9653 INFO sahara.plugins.base [-] Plugin storm loaded sahara.plugins.storm.plugin:StormProvider
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.462 9653 INFO sahara.plugins.base [-] Plugin spark loaded sahara.plugins.spark.plugin:SparkProvider
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 CRITICAL sahara [-] ConfigurationError: Plugins couldn't be loaded: hdp
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: Error ID: dacaaf79-7dca-40fc-b19f-573d9755d82e
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara Traceback (most recent call last):
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/bin/sahara-api", line 10, in <module>
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara sys.exit(main())
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/cli/sahara_api.py", line 53, in main
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara app = setup_api()
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/cli/sahara_api.py", line 43, in setup_api
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara server.setup_common(possible_topdir, 'API')
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/main.py", line 91, in setup_common
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara plugins_base.setup_plugins()
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/plugins/base.py", line 163, in setup_plugins
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara PLUGINS = PluginManager()
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/plugins/base.py", line 85, in __init__
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara self._load_cluster_plugins()
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara File "/usr/lib/python2.7/site-packages/sahara/plugins/base.py", line 111, in _load_cluster_plugins
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara ", ".join(requested_plugins - loaded_plugins))
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara ConfigurationError: Plugins couldn't be loaded: hdp
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara Error ID: dacaaf79-7dca-40fc-b19f-573d9755d82e
Aug 08 20:12:24 overcloud-controller-0 sahara-api[9653]: 2016-08-08 20:12:24.463 9653 ERROR sahara
Aug 08 20:12:24 overcloud-controller-0 systemd[1]: openstack-sahara-api.service: main process exited, code=exited, status=1/FAILURE
Aug 08 20:12:24 overcloud-controller-0 systemd[1]: Unit openstack-sahara-api.service entered failed state.
Aug 08 20:12:24 overcloud-controller-0 systemd[1]: openstack-sahara-api.service failed.
Aug 08 20:13:24 overcloud-controller-0 systemd[1]: Stopped Sahara API Server.

After removing hdp plugin from /etc/sahara/sahara.conf sahara services start without any problem.

Default plugins in sahara now are: 'vanilla', 'spark', 'cdh', 'ambari' from https://github.com/openstack/sahara/blob/master/sahara/plugins/opts.py#L20

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/352598

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/352598
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=49ded9872304c29d5e47d43b705a39f5c1668c95
Submitter: Jenkins
Branch: master

commit 49ded9872304c29d5e47d43b705a39f5c1668c95
Author: Sagi Shnaidman <email address hidden>
Date: Mon Aug 8 23:50:03 2016 +0300

    Remove deprecated hdp plugin and add ambari

    HDP plugin was deprecated in newton and not supported in sahara
    replace it with supported ambari plugin

    Change-Id: I6998e3e1e1fe9a81c7941aec90c876ee133a601d
    Closes-bug: 1611107

[Luigi Toscano (ltoscano)]

I think the fix here is a workaround and the more generic bug that should be really fixed is " The list of enabled Sahara plugins should be configurable ": https://bugs.launchpad.net/tripleo/+bug/1560098

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.0.0.0b3 development milestone.