| 2014-05-02 18:18:34 |
Trevor McKay |
description |
The scheme that Sahara/Hadoop currently use to match credentials to swift paths at runtime demands that a swift data source URL be in the form
"swift://container.sahara/object"
But, we don't force that during validation. This can result in a job that is launched and fails at runtime with "KILLED" -- there may or may not be useful error information in the top level Oozie log. A user may have to drill down into task logs to find out what happened.
We need to disallow swift paths that do not conform |
The scheme that Sahara/Hadoop currently use to match credentials to swift paths at runtime demands that a swift data source URL be in the form
"swift://container.sahara/object"
But, we don't force that during validation. This can result in a job that is launched and fails at runtime with "KILLED" -- there may or may not be useful error information in the top level Oozie log. A user may have to drill down into task logs to find out what happened.
We need to make sure that swift urls conform:
* require "swift" as the scheme if the data source type is "swift"
* require the ".sahara" suffix on the container
* require a non-zero length object |
|
