Missing config-flags

[Florian Haas]

This charm is sorely missing a generic config-flags option, as seen in the nova-compute or glance charms.

Right now, it's impossible to modify ceph.conf beyond auth_supported, {cluster,public}_network and fsid. Rather than implementing specific charm options for the usual suspects (osd_pool_default_size, osd_pool_default_min_size, osd_recovery_max_active, osd_max_backfills, mon_osd_down_out_interval come to mind) it would be way better to include a "config-flags" option, preferably not a flat dictionary but one that supports ceph.conf configuration sections like global, mon, osd etc.

[Florian Haas (fghaas)]

Possibly related to #1517846.

[Edward Hope-Morley (hopem)]

I think this is a valid request but it also presents us with a challenge because, as a rule, we try to avoid having charms offer one to one config mappings to their underlying services/apps in favour of having charms make informed decisions about how best to configure applications opaquely. My other general concern is that the more we provide users with ways to customise their configs, the further we get from ensuring that community knowledge is fed back into the charms for others to benefit from it and protect against potential configuration conflicts.

With applications like Ceph there is inevitably a requirement to adjust/adapt configurations over time and since Ceph provides such a rich set set of configurability there is wide net to support. Juju charms support something called 'actions' which allow developers to capture administrative operations into an application which can be called ad lib from the cli. While these are a nice way to perform complex admin tasks in a simple way and without having to expose too much config to the user, I think it is going to be hard to have it meet the requirement of supporting tweaking any parameter and covering any possible deployment scenario (although the most useful and common should definitely be catered for with actions and work is going on this cycle to address this). So I guess the challenge is to figure out whether we want to support this and if so, what's the happy medium.

[Florian Haas (fghaas)]

I'm sorry to report that that notion makes no sense. Ceph has over 800 config options; the idea that you'll cover them all with "informed decisions about how best to configure applications opaquely" looks ludicrous to me. But if you insist on trying, don't let me stop you.

Also, if your reasoning holds true; you'll have to drop your config-flags options from glance and nova-compute promptly.

[Chris Holcombe (xfactor973)]

Florian: please practice the code of conduct outlined here: http://www.ubuntu.com/about/about-ubuntu/conduct

[James Page (james-page)]

I'd be supportive of having a config-flags type of configuration option but what I would ask is that end-users of these flags document how they are using them and provide that as feedback to the charm development team.

I'd like to try to condense common use of such flags into higher level configuration options which then don't require expert knowledge to implement, and can be updated in the charm codebase as underlying implementation details change between ceph releases.

We've taken a similar approach to config-flags for other openstack related charms and its driven out some nice higher level features which are more generally accessible.

[Florian Haas (fghaas)]

Chris, from your comment I am assuming that you consider one of my comments, or the original bug report, to be in violation of the Ubuntu CoC. Please be specific and point out which part (or parts) you consider a violation (or violations), and also which part of the CoC you consider them to be in violation of. Thank you in advance.

[Florian Haas (fghaas)]

James, several options come to mind, already specified in the original report:

- osd_pool_default_size (default 3, frequently overridden as 2)
- osd_pool_default_min_size (default 2, frequently overridden as 1)
- osd_recovery_max_active (default 3 as of Jewel, 15 prior, frequently overridden as the pre-Jewel default is suboptimal)
- osd_max_backfills (default 1 as of Jewel, 10 prior, idem)
- mon_osd_down_out_interval (default 300, frequently overridden on multi-disk servers where a cold reboot including POST may be significantly longer than 300s)

But, as pointed out earlier, I consider wrapping these in separate charm options to be a bad idea. There are just too many config options to begin with, and predicting which of them will and will not be set is almost impossible, as this is highly dependent on the available hardware.

[James Page (james-page)]

Well that sounds like a good set of core options that are changed frequently to pull up into specific configuration options; I appreciate we won't every cover everything and I don't want todo that - that's not the objective we have for charms.

So like I said, we will look to add a config-flags style config option so expert users who know what they are doing can inject arbitrary configuration into ceph.conf - but it will come with an appropriate health warning :-). I'd rather do that than have end-users fork charms to achieve the same thing.

We can test what we can support with specific configuration options; but we can't make that guarantee with config-flags injected values - hence the health warning.

[Edward Hope-Morley (hopem)]

Fix proposed to branch: master
Review: https://review.openstack.org/321006/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321006
Committed: https://git.openstack.org/cgit/openstack/charm-ceph/commit/?id=af7ac0be617c6ed5b08c7bf37d33ace6749963b9
Submitter: Jenkins
Branch: master

commit af7ac0be617c6ed5b08c7bf37d33ace6749963b9
Author: Edward Hope-Morley <email address hidden>
Date: Mon May 2 12:34:45 2016 -0500

    Add support for user-provided ceph config

    Adds a new config-flags option to the charm that
    supports setting a dictionary of ceph configuration
    settings that will be applied to ceph.conf.

    This implementation supports config sections so that
    settings can be applied to any section supported by
    the ceph.conf template in the charm.

    Change-Id: I5ed2530b1e06a4565029d62124b469b97e17d342
    Closes-Bug: 1522375

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/321531

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/321545

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/321567

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/323782

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/323782
Committed: https://git.openstack.org/cgit/openstack/charm-ceph/commit/?id=a8594a9a43f4159643d3197cc1601c864729b8cc
Submitter: Jenkins
Branch: master

commit a8594a9a43f4159643d3197cc1601c864729b8cc
Author: Edward Hope-Morley <email address hidden>
Date: Wed Jun 1 11:37:10 2016 +0100

    Sync c-h and fix config-flags permitted sections

    Change-Id: I0f93d339f97108568c2c9afdf9dedc788b945c86
    Related-Bug: 1522375

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321567
Committed: https://git.openstack.org/cgit/openstack/charm-ceph-radosgw/commit/?id=485868fa543d2d9a2c8fa2fc19c07159aa37d9af
Submitter: Jenkins
Branch: master

commit 485868fa543d2d9a2c8fa2fc19c07159aa37d9af
Author: Edward Hope-Morley <email address hidden>
Date: Thu May 26 15:18:19 2016 +0100

    Add support for user-provided ceph config

    Adds a new config-flags option to the charm that
    supports setting a dictionary of ceph configuration
    settings that will be applied to ceph.conf.

    This implementation supports config sections so that
    settings can be applied to any section supported by
    the ceph.conf template in the charm.

    Change-Id: I8a447209b9040890e7c10585321b71da08a26b11
    Closes-Bug: 1522375

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321545
Committed: https://git.openstack.org/cgit/openstack/charm-ceph-osd/commit/?id=8f0347d69233bb7fae390dd35d0a03e586948a14
Submitter: Jenkins
Branch: master

commit 8f0347d69233bb7fae390dd35d0a03e586948a14
Author: Edward Hope-Morley <email address hidden>
Date: Thu May 26 14:48:16 2016 +0100

    Add support for user-provided ceph config

    Adds a new config-flags option to the charm that
    supports setting a dictionary of ceph configuration
    settings that will be applied to ceph.conf.

    This implementation supports config sections so that
    settings can be applied to any section supported by
    the ceph.conf template in the charm.

    Change-Id: I306fd138820746c565f8c7cd83d3ffcc388b9735
    Closes-Bug: 1522375

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321531
Committed: https://git.openstack.org/cgit/openstack/charm-ceph-mon/commit/?id=7fdc5b52c51ddbe3f44d0de994aa81d9cf6342af
Submitter: Jenkins
Branch: master

commit 7fdc5b52c51ddbe3f44d0de994aa81d9cf6342af
Author: Edward Hope-Morley <email address hidden>
Date: Thu May 26 14:14:22 2016 +0100

    Add support for user-provided ceph config

    Adds a new config-flags option to the charm that
    supports setting a dictionary of ceph configuration
    settings that will be applied to ceph.conf.

    This implementation supports config sections so that
    settings can be applied to any section supported by
    the ceph.conf template in the charm.

    Includes some unit test code cleanup.

    Closes-Bug: 1522375
    Change-Id: I5fa0890d87425499dbd48af6f2bc1f196713a975