EDP jobs execution fails using vanilla hadoop

[Shi Yan]

Using sahara-tests to run the scenario test on the pike version sahara, but the EDP job execution fails for most of them.

sahara-tests run below the six jobs, and the cluster creation and verification is successful:
pig_job mapreduce_job mapreduce_streaming_job
java_job
hive_job
spark_wordcount

only java job is succeeded, others get killed or error. Three types of errors from oozie are like below:

2017-12-13 00:57:17,956 WARN MapReduceActionExecutor:523 - SERVER[vanilla-271-24515547-master-oo-hs-sn-3af62c1b-0.novalocal] USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000002-171213005432651-oozie-hado-W] ACTION[0000002-171213005432651-oozie-hado-W@job-node] Launcher exception: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem not found
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem not found

2017-12-13 00:59:09,140 WARN PigActionExecutor:523 - SERVER[vanilla-271-24515547-master-oo-hs-sn-3af62c1b-0.novalocal] USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000003-171213005432651-oozie-hado-W] ACTION[0000003-171213005432651-oozie-hado-W@job-node] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]

2017-12-13 00:57:43,741 WARN HiveActionExecutor:523 - SERVER[vanilla-271-24515547-master-oo-hs-sn-3af62c1b-0.novalocal] USER[hadoop] GROUP[-] TOKEN[] APP[job-wf] JOB[0000001-171213005432651-oozie-hado-W] ACTION[0000001-171213005432651-oozie-hado-W@job-node] Launcher exception: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Tried to run the pig script manually on the existing cluster, the result is same. (Launcher error then get kiiled)

[Luigi Toscano (ltoscano)]

This seems interesting: org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem not found

Is the image generated using sahara-image-elements?

[Shi Yan (yanshi-403)]

yes, I am using the image generated from sahara-image-elements but with below change in diskimage-create.sh:

export DIB_HDFS_LIB_DIR="/opt/hadoop/share/hadoop/tools/lib"

Otherwise, the hadoop cluster will fail to start the datanode.