Juju requires creating webhook prefixed with model name

[Kenneth Koski]

I have a charm that deploys a workload that acts as a mutating and validating webhook. If the workload pod doesn't see the mutatingwebhookconfiguration or validatingwebhookconfiguration resources when it boots up, it will create them itself, but can't set them up correctly, as it doesn't have CA info for an HTTPS connection. The issue is that I can't prevent this from happening, as I'm unable to create a mutating/validating without the model name prefixed to it.

As a concrete example, the katib-controller workload pod will boot up, and check to see if mutatingwebhookconfiguration/katib-mutating-webhook-config exists. If it doesn't, it creates it with invalid data. I then tried to create this resource myself with the "mutatingwebhookconfiguration" field of kubernetesResources in the operator pod, so that the workload pod would see that it already exists and skip creating it. However, Juju created mutatingwebhookconfiguration/kubeflow-katib-mutating-webhook-config, which means that the workload pod went and created mutatingwebhookconfiguration/katib-mutating-webhook-config, and now I have two of those resources, one of which is broken.

[Ian Booth (wallyworld)]

The issue is that webhooks are global resources. Unless we prefix with the model name, it's possible to deploy 2 bundles to different models and they will overwrite each other's resources.

The correct juju way to do this is to use relation data to coordinate between workloads instead of looking for hard coded data.

[Kenneth Koski (knkski)]

Unfortunately there's not much we can do as far as coercing the workload pod into the juju way. Right now, I'm solving it by `kubectl delete`ing the workload-created webhooks. Is there any way we could handle that better in Juju?

[Ian Booth (wallyworld)]

We can look at providing an optional name attribute to override what Juju does to disambiguate the name used. It's better than kubectl deleting stuff but long term we'd want to fix upstream for sure.

[Yang Kelvin Liu (kelvin.liu)]

https://github.com/juju/juju/pull/11490 will be landed to 2.8 to fix this.