Add support to schedule by weight

[Alvaro Uria]

When reading about the Nova scheduler service, we found that hosts can be selected after applying filters and weights. By applying filters, hosts will be removed from the pool of available nodes, while weights will order a list by priority.

When a set of hypervisors are unbalanced due to IOPS, we think it is a better solution to deprioritize rather than remove from said pool of available hosts (user would get "no valid host was found" vs. a possibly slow environment).

In charm-nova-cloud-controller, "scheduler-default-filters" can be used to set filters. Weights use multiple configuration parameters on the "[filter_scheduler]" section [2]. I think a new config parameter similar to "config-flags" should be added which would support adding key=value parameters to said section ("filter-scheduler-config-flags").

1. https://docs.openstack.org/nova/latest/user/filter-scheduler.html
2. https://docs.openstack.org/nova/latest/user/filter-scheduler.html#weights

[Paul Goins (vultaire)]

FTR, the following is somewhat of a duplicate of this: https://bugs.launchpad.net/charm-nova-cloud-controller/+bug/1962544

[Paul Goins (vultaire)]

I have a customer who likely will need to be able to set ram_weight_multiplier and/or scheduler_host_subset_size. On Ussuri this looks like it'll work, but I see that these options are different in latest, both being moved to [filter_scheduler] instead of [DEFAULT].

This means that I will be able to use the config-flags setting to enable this for Ussuri, but if this customer upgrades to the next LTS when fully available and tested, this functionality will break for them unless we have some way of controlling these settings under the [filter_scheduler] group instead.