Comment 4 for bug 1817624

The solution proposed in comment #2 won't really work since get-addon-templates runs when the cdk-addons snap is built, so it wouldn't be able to alter the templates at charm run time.

We don't have a good way to customize pieces of the default CDK addon templates while retaining CDK management of those addons. In other words, it's all or nothing. You either accept the defaults, and CDK will make sure those pods are running, and that they get upgraded with new releases of CDK, or you turn off the addons that you want to customize and apply your own yaml. One problem with that is that you'd be responsible for upgrading versions of those addon templates yourself.

I believe the correct long term solution is that we should have a grafana k8s charm, on which you could set this config, which would in turn modify the pod spec. But we don't have that today.

One possible solution that falls between the ideal of the future and the restrictive reality of today is to add charm config on kubernetes-master through which the addons yaml could be customized. It might look something like this:

addon-customisations:
  kubernetes-dashboard: <custom yaml>
  influxdb-grafana-controller: <custom yaml>
  grafana-service: <none>
  ...

So basically one entry in the map for each template that CDK deploys, with optional yaml that would be merged onto the default. This would allow for customisations while retaining the ability for CDK to upgrade the default addon templates over time.

What do you think of this idea? Not sure how quickly we could get this done, but interested in feedback on the approach.

In the meantime, if you want to set this custom grafana config today, here's what you could do:

1) juju run --unit $(juju status|grep kubernetes-master|grep '*'|awk -F'*' '{print $1}') -- "for file in kubernetes-dashboard influxdb-grafana-controller influxdb-service grafana-service heapster-rbac heapster-controller heapster-service; do cat /root/snap/cdk-addons/current/addons/\${file}.yaml;echo '---'; done" > dashboard.yaml
2) edit dashboard.yaml to match desired input
3) juju config kubernetes-master enable-dashboard-addons=false
4) kubectl apply -f dashboard.yaml