The max_users_per_team setting can't be increased beyond 1000 if use_canonical_defaults is True

[Barry Price]

Modifying the charm setting `max_users_per_team` *ought* to update the pod spec and spawn a new pod with the new value in place.

Having just attempted this though, nothing appears to have happened, and the running unit and pod appear to have been unaffected.

Manually editing the deployment (e.g. `kubectl edit deployment mattermost`) shows the old value (for MM_TEAMSETTINGS_MAXUSERSPERTEAM) in place of the new one.

Modifying the value to the desired number within that `kubectl edit` session and saving it to apply seems to be a valid short-term workaround, but this can then come undone later when the charm notices the discrepancy.

Unsetting `use_canonical_defaults` would unfortunately introduce other operational changes which can't be worked around via current config options.

[Barry Price (barryprice)]

Okay, I've dug into the code and it appears we do read this value from config, but then override it via the _update_pod_spec_for_canonical_defaults method if `use_canonical_defaults` is set:

https://git.launchpad.net/charm-k8s-mattermost/tree/src/charm.py#n399

I think we need to either rethink that, or increase the default value from 1000 to say 2000 - 1000 is no longer enough for our deployments.