Comment 1 for bug 1470841

Revision history for this message
Elise Gafford (egafford) wrote :

From http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.3.3/bk_using_Ambari_book/content/install-ha_2x.html:

   If you are using Oozie, you need to use the Nameservice URI instead of the NameNode URI in your workflow files. For example, where the Nameservice ID is mycluster:

  <workflow-app xmlns="uri:oozie:workflow:0.2" name="map-reduce-wf">
      <start to="mr-node"/>
      <action name="mr-node">
          <map-reduce>
              <job-tracker>${jobTracker}</job-tracker>
              <name-node>hdfs://mycluster</name-node>

From http://172.24.4.230:8080/#/main/hosts/my-hdp2ha-b090a511-master-ha-common-f2ebfbc6-001.novalocal/configs (which pulls from hdfs-site.xml):

  dfs.nameservices: my-hdp2ha-b090a511

From your cluster definition (which should be valid):
  | info | {u'HDFS': {u'NameNode': |
  | | u'hdfs://172.24.4.229:8020', u'Web UI': |
  | | u'http://172.24.4.229:50070'}, u'JobFlow': |
  | | {u'Oozie': u'http://172.24.4.230:11000'}, |
  | | u'MapReduce2': {u'Web UI': |
  | | u'http://172.24.4.230:19888', u'History |
  | | Server': u'172.24.4.230:10020'}, u'Yarn': |
  | | {u'Web UI': u'http://172.24.4.230:8088', |
  | | u'ResourceManager': u'172.24.4.230:8050'}, |
  | | u'Ambari Console': {u'Web UI': |
  | | u'http://172.24.4.230:8080'}} |

From sahara/sahara/service/edp/oozie/engine.py:
        nn_path = self.get_name_node_uri(self.cluster)
        ...
        job_parameters = {
            "jobTracker": rm_path,
            "nameNode": nn_path,
            "user.name": hdfs_user,
            oozie_libpath_key: oozie_libpath,
            app_path: "%s%s" % (nn_path, path_to_workflow),
            "oozie.use.system.libpath": "true"}

From sahara/sahara/plugins/hdp/edp_engine.py:
    def get_name_node_uri(self, cluster):
        return cluster['info']['HDFS']['NameNode']

It seems that we've succeeded in setting up a highly available cluster, but we're hard-coding ourselves into only using Oozie through one of the nodes, rather than using the nameservice; as the [info][HDFS][NameNode] is hardcoded as of cluster creation time, and the feature did not build any sense of nameservice into Sahara.