[DOC] Features overview suggests Cinders to increase reliability

Bug #1371360 reported by Andrew Lazarev on 2014-09-19
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fix Released
Nikita Konovalov

Bug Description

"Using Cinder volumes increases reliability of data which is important for HDFS service."

This is not true. We strongly not recommend to use any Cinder devices with replication enabled. HDFS makes replication by itself and replication on device level will multiply number of chunks one more time. So, if HDFS has replication factor equal to 3 and Cinder configured to use Swift with replication factor 3, resulting replication factor will be 9. That's why we recommend using Cinder for devices that don't replicate data. This is not something called 'reliability'.

Need to describe this in docs.

Changed in sahara:
status: New → Confirmed
tags: added: docs
Changed in sahara:
importance: Undecided → Medium
milestone: none → juno-rc1
Changed in sahara:
assignee: nobody → Sharan Kumar (sharan-monikantan)
Changed in sahara:
status: Confirmed → In Progress
Changed in sahara:
milestone: juno-rc1 → 2014.2
Duncan Thomas (duncan-thomas) wrote :

Cinder does *not* use swift, except for backup.

I'm not sure that this bug makes any sense. The use of replication in cinder is designed for disaster recovery type scenarios, and currently has little effect on per-volume reliability.

John Griffith (john-griffith) wrote :

Hey Andrew, maybe a link to the doc would help here? I suspect there's an overloaded term here somewhere and we just need to be more explicit in said doc.

Changed in sahara:
milestone: 2014.2 → juno-rc1
Changed in sahara:
milestone: juno-rc1 → kilo-1
Changed in sahara:
milestone: kilo-1 → kilo-2
Changed in sahara:
assignee: Sharan Kumar (sharan-monikantan) → nobody
importance: Medium → Low
status: In Progress → Confirmed
Changed in sahara:
milestone: kilo-2 → kilo-3
Changed in sahara:
milestone: kilo-3 → liberty-1
plieb (jliberma) on 2015-03-18
Changed in sahara:
assignee: nobody → plieb (jliberma)
Daniele Venzano (venza) wrote :

Using Cinder for HDFS has performance issues that are not well understood.
Once you lose data locality, computation will no longer happen near the data and jobs will make heavy use of the network. Many of the assumptions that Hadoop/Spark make are no longer valid when HDFS is backed by network volumes.

While Sahara supporting Cinder is a good thing, I would be wary of even suggesting such a solution without a well-thought document of the pros and cons that people can read and understand.

Changed in sahara:
milestone: liberty-1 → liberty-2
Thierry Carrez (ttx) on 2015-07-28
Changed in sahara:
milestone: liberty-2 → liberty-3
Changed in sahara:
milestone: liberty-3 → liberty-rc1
Changed in sahara:
milestone: liberty-rc1 → next
Sergey Reshetnyak (sreshetniak) wrote :

This bug is > 60 days without activity. We are unsetting assignee and milestone and setting status to Incomplete in order to allow its expiry in 60 days.

If the bug is still valid, then update the bug status.

Changed in sahara:
assignee: plieb (jliberma) → nobody
milestone: next → none
status: Confirmed → Incomplete
Changed in sahara:
status: Incomplete → Triaged
assignee: nobody → Nikita Konovalov (nkonovalov)
milestone: none → mitaka-rc1

Fix proposed to branch: master
Review: https://review.openstack.org/292859

Changed in sahara:
status: Triaged → In Progress
Changed in sahara:
milestone: mitaka-rc1 → newton-1

Reviewed: https://review.openstack.org/292859
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=1af4701b854d14134e5b9fdba6ba73f5326c361b
Submitter: Jenkins
Branch: master

commit 1af4701b854d14134e5b9fdba6ba73f5326c361b
Author: Nikita Konovalov <email address hidden>
Date: Tue Mar 15 15:30:48 2016 +0300

    Updated volumes section in docs

    Improved description for cinder volumes usage mechanism.
    Described the volumes backed by distributed storage case.

    Change-Id: Ie5a436daaca9e5ed009aa131cf6487be3057182a
    Closes-bug: #1371360

Changed in sahara:
status: In Progress → Fix Released

This issue was fixed in the openstack/sahara development milestone.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers