customize-failure-domain only works when enabled both in ceph-mon and ceph-osd

Bug #1752032 reported by Sandor Zeestraten
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceph Monitor Charm
Triaged
Low
Unassigned
Ceph OSD Charm
Triaged
Low
Unassigned

Bug Description

The customize-failure-domain configuration option only creates a correct crush map when it is enabled both in the ceph-mon and ceph-osd charm. If it is enabled in only ceph-mon or ceph-osd, you end up with undesirable crush maps. The docs for this configuration option (both for ceph-mon and ceph-osd) do not communicate this.

Suggested fixes:
* Remove the configuration option available from either ceph-mon or ceph-osd to avoid confusion.
* Fix the docs and explain what it is actually doing underneath so people don't get caught out.

Below is some info from a reproduction:
3x juju availility zones
3x ceph-mon
3x ceph-osd with 2x HDD each

# Enabled in ceph-mon
Bundle: https://pastebin.com/raw/3cdRDJpp
Crush map: https://pastebin.com/raw/T5X01up8
Result: Missing the correct buckets (default being the rack type) for each juju AZ.

# Enabled in ceph-osd
Bundle: https://pastebin.com/raw/8PD2Ky4S
Crush map: https://pastebin.com/raw/Uc3FHWpa
Result: Correct buckets for each juju AZ, however it has the wrong failure domain for the ruleset, i.e. "step chooseleaf firstn 0 type host" instead of "step chooseleaf firstn 0 type rack". See also https://youtu.be/-FOYXz3Bz3Q?t=83

# Enabled in both ceph-mon and ceph-osd
Bundle: https://pastebin.com/raw/wy95UASm
Crush map: https://pastebin.com/raw/MK4wxspe
Result: Correct crush map with rack buckets for each juju AZ and correct failure domain in ruleset.

# Versions
juju 2.1.2
maas 2.1.3
ceph-mon rev. 20
ceph-osd rev. 255

Tags: docs
description: updated
James Page (james-page)
Changed in charm-ceph-mon:
status: New → Triaged
Changed in charm-ceph-osd:
status: New → Triaged
Changed in charm-ceph-mon:
importance: Undecided → Low
Changed in charm-ceph-osd:
importance: Undecided → Low
tags: added: docs
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.