It looks like this error message is somewhat misleading, and actually covering up another issue. I wasn't using pod spec v2 correctly, and the CRDs for each service were not getting created, which was causing the workload pods to get constantly rebooted.
However, now that I've created the CRDs manually, I'm still seeing an issue. "juju status" will show a service as active and ready, and then eventually go to the `creating or updating service account: role binding "FOO" already exists` error, and then cycle back to active and ready. The rolebindings appear to be getting constantly deleted and recreated, though I'm not sure what the timings are relative to the error message that I'm getting. An example rolebinding:
It looks like this error message is somewhat misleading, and actually covering up another issue. I wasn't using pod spec v2 correctly, and the CRDs for each service were not getting created, which was causing the workload pods to get constantly rebooted.
However, now that I've created the CRDs manually, I'm still seeing an issue. "juju status" will show a service as active and ready, and then eventually go to the `creating or updating service account: role binding "FOO" already exists` error, and then cycle back to active and ready. The rolebindings appear to be getting constantly deleted and recreated, though I'm not sure what the timings are relative to the error message that I'm getting. An example rolebinding:
Name: pipelines- api-pipeline- runner pipelines- api
juju-model= kubeflow
Labels: juju-app=
Annotations: <none>
Role:
Kind: ClusterRole
Name: pipeline-runner
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount pipelines-api kubeflow