CRD creation fails in pod spec charms on juju 2.9.25

[Dominik Fleischmann]

When deploying a pod-spec charm that also creates a CustomResourceDefinition, on Kuberentes 1.21 with juju 2.9.25 the following error occurs:

creating or updating custom resource definitions: ensuring custom resource definition "authcodes.dex.coreos.com" with version "v1beta1": CustomResourceDefinition.apiextensions.k8s.io "authcodes.dex.coreos.com" is invalid: spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required

Steps to reproduce:
sudo snap install microk8s --classic --channel=1.21/stable
microk8s enable dns storage ingress metallb:10.64.140.43-10.64.140.49 # probably also happens with only dns and storage
juju bootstrap microk8s
juju add-model kubeflow
juju deploy dex-auth

This is currently blocking everyone from deploying Kubeflow on the latest version of juju.

[Rob Gibbon (robgibbon)]

https://github.com/juju/juju/commit/695d9bc09df0725a168df52707336bb6a3a92ff7

I think this commit might be what broke it. In caas/kubernetes/provider/customresourcedefinitions.go you add some logic that converts a v1beta1 CRD into a v1 CRD but this I think won't work on K8s 1.21, which Kubeflow requires.

```go
  var err error
  switch v.Spec.Version {
  case k8sspecs.K8sCustomResourceDefinitionV1:
   if k8sVersion.Major == 1 && k8sVersion.Minor < 16 {
    return cleanUps, errors.NotSupportedf("custom resource definition version %q", v.Spec.Version)
   } else {
    out, crdCleanUps, err = k.ensureCustomResourceDefinitionV1(obj, v.Spec.SpecV1)
   }
  case k8sspecs.K8sCustomResourceDefinitionV1Beta1:
   if k8sVersion.Major == 1 && k8sVersion.Minor < 16 {
    out, crdCleanUps, err = k.ensureCustomResourceDefinitionV1beta1(obj, v.Spec.SpecV1Beta1)
   } else {
    var newSpec apiextensionsv1.CustomResourceDefinitionSpec
    newSpec, err = k8sspecs.UpgradeCustomResourceDefinitionSpecV1Beta1(v.Spec.SpecV1Beta1)
    if err != nil {
     err = errors.Annotatef(err, "cannot convert v1beta1 crd to v1")
     break
    }
    out, crdCleanUps, err = k.ensureCustomResourceDefinitionV1(obj, newSpec)
   }
  default:
   // This should never happen.
   return cleanUps, errors.NotSupportedf("custom resource definition version %q", v.Spec.Version)
```

[Harry Pidcock (hpidcock)]

https://github.com/juju/juju/pull/13758