[Sahara][CDH] Unable to launch Spark job with Swift

[Vitalii Gridnev]

Upstream bug: https://bugs.launchpad.net/sahara/+bug/1474128

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-7.0/2015.1.0
Change author: Oleg Borisenko <email address hidden>
Review: https://review.fuel-infra.org/9968

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9968
Submitter: mos-infra-ci <>
Branch: openstack-ci/fuel-7.0/2015.1.0

Commit: 967da9f11023a4ad4b2deb4708ca37d6c6ee2a04
Author: Oleg Borisenko <email address hidden>
Date: Wed Jul 29 19:55:58 2015

EDP Spark jobs work with Swift

1) Fixed the path to hadoop-swift.jar - in cloudera
   it's named as hadoop-openstack.jar
2) Fixed the options for launch wrapper with yarn-cluster
   (more details at http://spark.apache.org/docs/latest/running-on-yarn.html
    'Important notes' section).
3) Fixed the issue of swift credentials visibility in Yarn cluster.
4) Fixed related unit-test with the same error in it.

Change-Id: I5e8c72f0e362792f06245b3744a32342abc42389
Closes-bug: 1474128
Closes-bug: 1479404
(cherry picked from commit 05244cbd490a5f0226d023ada9f3b74b22e1ac72)

[Vitalii Gridnev (vgridnev)]

verified on iso 288

Steps to verify:

Create cluster with one node group that contains the following node processes:

CLOUDERA_MANAGER
YARN_JOBHISTORY
OOZIE_SERVER
HDFS_DATANODE
YARN_RESOURCEMANAGER
HDFS_SECONDARYNAMENODE
HDFS_NAMENODE
SPARK_YARN_HISTORY_SERVER
YARN_NODEMANAGER

and then launch spark job that tries to access to swift.