[hadoop-swift] Driver fails to read data from swift if destination is not in swift

[Andrew Lazarev]

Steps to reproduce

1. Install hadoop cluster by savanna
2. ssh to worker
3. try to copy file from swift to HDFS

Stacktrace:

2014-02-26 00:06:10,176 DEBUG org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem: Initializing SwiftNativeFileSystem against URI swift://demo.savanna/input and working dir swift://demo.savanna/user/hadoop
2014-02-26 00:06:10,204 DEBUG org.apache.hadoop.fs.swift.http.RestClientBindings: Filesystem swift://demo.savanna/input is using configuration keys fs.swift.service.savanna
2014-02-26 00:06:10,206 DEBUG org.apache.hadoop.fs.swift.http.SwiftRestClient: Service={savanna} container={demo} uri={http://172.18.168.2:5000/v2.0/tokens/} tenant={demo} user={demo-user} region={(none)} publicURL={true} location aware={false} partition size={4718592 KB}, buffer size={64 KB} block size={32768 KB} connect timeout={15000}, retry count={3} socket timeout={60000} throttle delay={0}
2014-02-26 00:06:10,206 DEBUG org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem: SwiftFileSystem initialized
2014-02-26 00:06:10,210 DEBUG org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore: Reading swift://demo.savanna/input from proxy node
2014-02-26 00:06:10,217 INFO org.apache.hadoop.tools.DistCp: FAIL input : org.apache.hadoop.fs.swift.exceptions.SwiftInternalStateException: Null Endpoint -client is not authenticated
 at org.apache.hadoop.fs.swift.http.SwiftRestClient.checkNotNull(SwiftRestClient.java:1800)
 at org.apache.hadoop.fs.swift.http.SwiftRestClient.pathToURI(SwiftRestClient.java:1629)
 at org.apache.hadoop.fs.swift.http.SwiftRestClient.pathToURI(SwiftRestClient.java:1669)
 at org.apache.hadoop.fs.swift.http.SwiftRestClient.getData(SwiftRestClient.java:711)
 at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.getObject(SwiftNativeFileSystemStore.java:294)
 at org.apache.hadoop.fs.swift.snative.SwiftNativeInputStream.<init>(SwiftNativeInputStream.java:103)
 at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem.open(SwiftNativeFileSystem.java:555)
 at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem.open(SwiftNativeFileSystem.java:536)
 at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:419)
 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:547)
 at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:314)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
 at org.apache.hadoop.mapred.Child.main(Child.java:249)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/76405

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/76405
Committed: https://git.openstack.org/cgit/openstack/savanna-extra/commit/?id=fab9840abf46f3867eba81ee95b3b3cbda2124ac
Submitter: Jenkins
Branch: master

commit fab9840abf46f3867eba81ee95b3b3cbda2124ac
Author: Andrew Lazarev <email address hidden>
Date: Tue Feb 25 16:58:49 2014 -0800

    [hadoop-swift] Added auth check before getData call

    Without such check client doesn't try to authenticate during method call.
    This leads to read failure in case if this is the first use of client.

    Change-Id: Ib2389a83b4e73fac49066ffe0411ff430482f848
    Closes-Bug: #1284906