Charm managed basic_auth.csv needs non-juju/ssh management method

[Drew Freiberger]

The management of basic_auth.csv for access to CDK dashboards/metrics via "juju ssh kubernetes-master/0" (or other leader unit) is unsustainable in a Managed Services Infrastructure environment. If your Kubernetes Cluster admins are not the same as the Infrastructure Admins (as in the case of Bootstack managed CDK environments), there is cross-over in separation of duties caused by this basic_auth authentication mechanism.

I see that there is the ability to relate to a Keystone infrastructure for external user management, but that is a bit heavy handed if not using an openstack environment as you must now have a database and rabbitmq environment stood up for keystone to be supported.

We need to solve managing the basic_auth.csv file without requiring ssh/root access to the underlying juju units.

Also, we need ways to ensure that the management of this file allows for undercloud users that should not be managable via the overcloud admins. For instance, we create a cloudadmin account for overcloud Customers, but retain the admin account for Bootstack usage. cloudadmin account should not be able to manage the admin account's entry in basic_auth.csv, as this is tied into juju access to kubernetes.

[Drew Freiberger (afreiberger)]

As a note, I had considered using juju actions for this, but that still doesn't honor the undercloud/overcloud privilege separation we use in our managed service offerings, however, an action would make it a much easier process for undercloud operators.

[Paul Goins (vultaire)]

I've noted that in production that basic_auth.csv appears to be frequently touched for some reason, roughly once every 5 minutes. If kube-apiserver gets restarted on the kubernetes-master nodes, additional logins may stop working.

That's what I presently see after a customer reported not being able to log in: admin credentials are preserved, but no other credentials are present in the file anymore. Since kube-apiserver was restarted, any credentials which might have been wiped from this file but still in memory are now gone.

[Paul Goins (vultaire)]

subscribed ~field-critical

[Paul Goins (vultaire)]

unsubscribed ~field-critical; broke my concerns out into a new ticket: https://bugs.launchpad.net/charm-kubernetes-master/+bug/1826260