config option min-cluster-size is not needed and confusing

[Nicolas Bock]

The configuration option `min-cluster-size` is unnecessary since there is charmhelpers.core.hookenv.expected_peer_units(). It is also confusing since it's not clear why someone would want to set its value to less than the number of deployed units.

The option should be removed and replaced internally by calls to charmhelpers.core.hookenv.expected_peer_units().

[1] https://charm-helpers.readthedocs.io/en/latest/api/charmhelpers.core.hookenv.html#charmhelpers.core.hookenv.expected_peer_units

[Chris MacNaughton (chris.macnaughton)]

Removing a config option entirely is something that is somewhat risky, and more so when the new charm hook (goal-state) is only ~two years old (https://juju.narkive.com/PQSrYowV/juju-2-4-beta1-is-released). The OpenStack charms generally (although not explicitly confirmed) should still support pre goal-state Juju. If we want to discuss breaking that support, then we can discuss removing config options that make clustering "safe" with pre goal-state Juju.

[Nicolas Bock (nicolasbock)]

I have looked through the charm and it's still not clear to me what the point of the option is. For example, if I add a unit to an existing cluster I will get a situation where I have N+1 nodes in the cluster from the point of view of RabbitMQ but min-cluster-size is still N. The same discrepancy occurs if I remove a unit. The operator has to remember to adjust min-cluster-size as the RabbitMQ cluster is scaled.

I understand your concern regarding the removal of a config option. We could phase it out more gradually instead. Would you be open to adding some logic to the charm such that it uses expected_peer_units where we use min-cluster-size right now? Or at least warn the operator that there is a mismatch between those two?

[Chris MacNaughton (chris.macnaughton)]

The config option isn't for handling post-deployment resize, it's for deferring cluster setup until the expected cluster size is met. When this isn't done, a single rabbit unit can come up and go: "Hey, I'm ready, let's send credentials down my relations", and then, a moment later, it's not ready because now it's doing cluster joining things, and then remote relations start failing when Rabbit isn't available

[Nicolas Bock (nicolasbock)]

Ok, that may be the intent, but how is this setting supposed to be used in a scale up/down scenario?