[hadoop-swiftfs] cannot read large object that uploaded by hadoop command

[Kazuki OIKAWA]

If large object uploaded by hadoop command, cannot read that object.
Because uploaded segment object name is wrong.

$ hadoop fs -copyFromLocal ./largeFile swift://test.sahara/large
$ hadoop fs -cat swift://test.sahara/large | wc -c
0
$ swift list test
large
swift://test.sahara/large/000001
swift://test.sahara/large/000002
swift://test.sahara/large/000003
swift://test.sahara/large/000004

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/162005

[Luigi Toscano (ltoscano)]

I see the the proposed fix is for sahara-extra. Does it affect the copy of hadoop-swiftfs upstream? If yes, is there a proposed patch against that version, or is there a plan to use the sahara-extra version always (if not already)?

[Kazuki OIKAWA (k.oikw)]

> Does it affect the copy of hadoop-swiftfs upstream?
yes

I recommend using sahara-extra version

I think we need to discuss swiftfs maintenance policy.

Currently, swiftfs of apache hadoop repository is hard to merge new patch.
Because swift understood reviewer is a few, maybe.

[Luigi Toscano (ltoscano)]

I understand the slowness of the review, but keeping a fork is always complicated. I would try to push the patch upstream anyway first. Maybe the maintainership of that component upstream can be extended to someone else too from "our" (Sahara) world to speed up things?

Moreover, right now the swiftfs_hadoop element from sahara-image-elements, which installs the Sahara version of the library if I'm not mistaken, is used only for vanilla and spark, so any change in swiftfs/hadoop library usage should impact that file as well.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/162005
Committed: https://git.openstack.org/cgit/openstack/sahara-extra/commit/?id=c88a30012dc4e451b9f72db2684691bb2111e7e7
Submitter: Jenkins
Branch: master

commit c88a30012dc4e451b9f72db2684691bb2111e7e7
Author: Kazuki OIKAWA <email address hidden>
Date: Fri Mar 6 13:11:42 2015 +0900

    Fix segment object naming that uploaded at qualified URL

    Change-Id: Ifdac5efe2c79a4ffed96f6fd66e58d2a263925fc
    Closes-Bug: 1428939