Spark plugin doesn't support several cinder volumes per node

[Sergey Reshetnyak]

Spark plugin incorrect uses cinder volumes.

Spark plugin create directories for cinder volumes. Command for create directories look at:

dn_path = c_helper.extract_hadoop_path(storage_paths, '/dfs/dn')
nn_path = c_helper.extract_hadoop_path(storage_paths, '/dfs/nn')
hdfs_dir_cmd = ('sudo mkdir -p %(nn_path)s %(dn_path)s &&'
'sudo chown -R hdfs:hadoop %(nn_path)s %(dn_path)s &&'
'sudo chmod 755 %(nn_path)s %(dn_path)s' %
{"nn_path": nn_path, "dn_path": dn_path})

'c_helper.extract_hadoop_path' function return string with comma separated patches, but need a string with space separated patches.

[Andrew Lazarev (alazarev)]

Actually this is not a single place where Spark plugin assumes that there is at most 1 volume.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/125970

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/125970
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=5aca5fe75aa94fe548fc0d481ad0a8edf7375d4f
Submitter: Jenkins
Branch: master

commit 5aca5fe75aa94fe548fc0d481ad0a8edf7375d4f
Author: Sergey Reshetnyak <email address hidden>
Date: Fri Oct 3 17:45:08 2014 +0400

    Fix working Spark with cinder volumes

    Change-Id: I4eb0d00766066f0d387e632b0217e11cfed552c5
    Closes-bug: #1376790

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: proposed/juno
Review: https://review.openstack.org/126387

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126387
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=ef312384f1a9fb202d2e2a784c9d9ece0cdcd93f
Submitter: Jenkins
Branch: proposed/juno

commit ef312384f1a9fb202d2e2a784c9d9ece0cdcd93f
Author: Sergey Reshetnyak <email address hidden>
Date: Fri Oct 3 17:45:08 2014 +0400

    Fix working Spark with cinder volumes

    Change-Id: I4eb0d00766066f0d387e632b0217e11cfed552c5
    Closes-bug: #1376790
    (cherry picked from commit 5aca5fe75aa94fe548fc0d481ad0a8edf7375d4f)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/128889

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/128889
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=2217fb27ecf8c5b4e4c4673b5f22f0f16016b677
Submitter: Jenkins
Branch: master

commit 3630ccffb25f66e2efc9297b0ecb852f8d932363
Author: Trevor McKay <email address hidden>
Date: Wed Oct 1 17:23:29 2014 -0400

    Fix HDFS url description, and other various edits

    HDFS url description is wrong as a result of code changes. This was
    the major motivation for this CR.

    Additional changes

    * formatted for 80 characters
    * consistent use of '.' at the end of bullets
    * added mention of Spark
    * adding '.sahara' suffix is no longer necessary
    * some other minor changes

    Closes-Bug: 1376457
    Change-Id: I72134bcdf6c42911d07e65952a9a56331d896699
    (cherry picked from commit a718ec7ddf85ef2e1e17868f6e2cd05b1c2762cd)

commit ff3bf76318821336810709eb1ff4b88cf94b67c7
Author: Trevor McKay <email address hidden>
Date: Wed Oct 1 13:16:57 2014 -0400

    Remove line saying that scaling and EDP are not supported for Spark

    Closes-Bug: 1376364
    Change-Id: I82249f8b9fb932c206876c2f6652c0a0b9e0650b
    (cherry picked from commit e385e3ed02bddf4db3f0b82c800b2cc0e2c056ba)

commit 4f23cfefa18332274d88475984491facd79b85f3
Author: Trevor McKay <email address hidden>
Date: Wed Oct 1 12:34:14 2014 -0400

    Description of job config hints in new doc page is wrong

    The 'configs' field is not a dictionary, it is actually
    a list of dictionaries. Update the description.

    Closes-Bug: #1357615
    Change-Id: I540abe050f1d81e36f4b5dcca547a7e5c3514c84
    (cherry picked from commit 61be4ece04d6370086d8b5b9bea4224010ec0d15)

commit 0d94b67fca6b0c5776ddcfe0f3e5b489afe376ea
Author: Michael McCune <email address hidden>
Date: Wed Oct 1 11:25:41 2014 -0400

    Removing extraneous Swift information from Features

    Changes
    * removing repeated information from Features page for Swift integration
    * refactoring features.rst to 80 columns

    Change-Id: Ib37e4476258cc4547d4a27847c89a9611bff05bc
    Closes-Bug: #1376309
    (cherry picked from commit eb529ca4f2dd153d494c4e02dd302998b3d6f43b)

commit 9e3fbb654d3530b11d3e6c1fb652028e631e5859
Author: Trevor McKay <email address hidden>
Date: Tue Sep 30 16:08:15 2014 -0400

    Update the Elastic Data Processing (EDP) documentation page

    * Add description of MapReduce.Streaming job type
    * Add description of Spark job type
    * Add reference to advanced configuration for Swfit proxy
    * Note that .sahara suffix is added to Swift URLs automatically
    * A few minor changes

    Closes-Bug: 1374574
    Closes-Bug: 1374606
    Change-Id: Ie53888975ce436439cc808b2fdc45dff66bae1a9
    (cherry picked from commit 7973db35e61b0c2d686798cb2de50d281713b03b)

commit 360aedfb323fb888acc4745b262eb7746d14ef27
Author: Trevor McKay <email address hidden>
Date: Tue Sep 30 12:51:08 2014 -0400

    Add documentation on the EDP job engine SPI

    Closes-Bug: 1357615
    Change-Id: I57dae10da9460deb2a332025cc3a0ea37ae233ee
    (cherry picked from commit 62ba37a8c415f1c422f010c96c0d553ff788d343)

commit 9fa0c5473d29c5eeeef3a23e7...