| 2018-02-27 09:35:04 |
Sandor Zeestraten |
description |
The customize-failure-domain configuration option only creates a correct crush map when it is enabled both in the ceph-mon and ceph-osd charm. If it is enabled in only ceph-mon or ceph-osd, you end up with undesirable crush maps. The docs for this configuration option (both for ceph-mon and ceph-osd) do not communicate this.
Suggested fixes:
* Remove the configuration option available from either ceph-mon or ceph-osd to avoid confusion.
* Fix the docs and explain what it is actually doing underneath so people don't get caught out.
Below is some info from a reproduction:
3x juju availility zones
3x ceph-mon
3x ceph-osd with 2x HDD each
# Enabled in ceph-mon
Bundle: https://pastebin.com/raw/3cdRDJpp
Crush map: https://pastebin.com/T5X01up8
Result: Missing the correct buckets (default being the rack type) for each juju AZ.
# Enabled in ceph-osd
Bundle: https://pastebin.com/raw/8PD2Ky4S
Crush map: https://pastebin.com/raw/Uc3FHWpa
Result: Correct buckets for each juju AZ, however it has the wrong failure domain for the ruleset, i.e. "step chooseleaf firstn 0 type host" instead of "step chooseleaf firstn 0 type rack". See also https://youtu.be/-FOYXz3Bz3Q?t=83
# Enabled in both ceph-mon and ceph-osd
Bundle: https://pastebin.com/raw/wy95UASm
Crush map: https://pastebin.com/raw/MK4wxspe
Result: Correct crush map with rack buckets for each juju AZ and correct failure domain in ruleset.
# Versions
juju 2.1.2
maas 2.1.3
ceph-mon rev. 20
ceph-osd rev. 255 |
The customize-failure-domain configuration option only creates a correct crush map when it is enabled both in the ceph-mon and ceph-osd charm. If it is enabled in only ceph-mon or ceph-osd, you end up with undesirable crush maps. The docs for this configuration option (both for ceph-mon and ceph-osd) do not communicate this.
Suggested fixes:
* Remove the configuration option available from either ceph-mon or ceph-osd to avoid confusion.
* Fix the docs and explain what it is actually doing underneath so people don't get caught out.
Below is some info from a reproduction:
3x juju availility zones
3x ceph-mon
3x ceph-osd with 2x HDD each
# Enabled in ceph-mon
Bundle: https://pastebin.com/raw/3cdRDJpp
Crush map: https://pastebin.com/raw/T5X01up8
Result: Missing the correct buckets (default being the rack type) for each juju AZ.
# Enabled in ceph-osd
Bundle: https://pastebin.com/raw/8PD2Ky4S
Crush map: https://pastebin.com/raw/Uc3FHWpa
Result: Correct buckets for each juju AZ, however it has the wrong failure domain for the ruleset, i.e. "step chooseleaf firstn 0 type host" instead of "step chooseleaf firstn 0 type rack". See also https://youtu.be/-FOYXz3Bz3Q?t=83
# Enabled in both ceph-mon and ceph-osd
Bundle: https://pastebin.com/raw/wy95UASm
Crush map: https://pastebin.com/raw/MK4wxspe
Result: Correct crush map with rack buckets for each juju AZ and correct failure domain in ruleset.
# Versions
juju 2.1.2
maas 2.1.3
ceph-mon rev. 20
ceph-osd rev. 255 |
|
