Cinder storage Class doesn't seem to be setting up volumes correctly

[Dominik Fleischmann]

I deployed a CDK on top of OpenStack with the following commands:

juju deploy kuberentes-core --overlay k8s-openstack-overlay.yaml

with the following k8s-openstack-overlay.yaml being:

applications:
  openstack-integrator:
    charm: cs:~containers/openstack-integrator
    options:
      subnet-id: "45fb4f09-f51c-475d-a13c-4549b6355e38"
    num_units: 1
relations:
  - ['openstack-integrator', 'kubernetes-master']
  - ['openstack-integrator', 'kubernetes-worker']

When I execute the following commands:

kubectl create -f - <<EOY
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: testclaim2
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
  storageClassName: cinder
EOY

kubectl create -f - <<EOY
apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
    - image: busybox
      command:
        - sleep
        - "3600"
      imagePullPolicy: IfNotPresent
      name: busybox
      volumeMounts:
        - mountPath: "/pv"
          name: testvolume
  restartPolicy: Always
  volumes:
    - name: testvolume
      persistentVolumeClaim:
        claimName: testclaim2
EOY

The busybox never gets to a ready state due to: 'pod has unbound immediate PersistentVolumeClaims'

This is using the cinder storage class that is created automatically when using the openstack-integrator charm.

If I create my own storage class like this

kubectl create -f - <<EOY
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: openstack-standard
provisioner: kubernetes.io/cinder
EOY

And I create the busybox with this storage class it works.

These are the descriptions of the 2 storage classes:
ubuntu@jujucli:~$ kubectl describe storageclass cinder
Name: cinder
IsDefaultClass: No
Annotations: <none>
Provisioner: csi-cinderplugin
Parameters: <none>
AllowVolumeExpansion: <unset>
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: Immediate
Events: <none>
ubuntu@jujucli:~$ kubectl describe storageclass openstack-standard
Name: openstack-standard
IsDefaultClass: No
Annotations: <none>
Provisioner: kubernetes.io/cinder
Parameters: <none>
AllowVolumeExpansion: <unset>
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: Immediate
Events: <none>

[Cory Johns (johnsca)]

The changes to CDK to use the external cloud provider for openstack, the cinder CSI plugin, and to automatically create the storage classes got missed for the latest release, so "kubernetes.io/cinder" is still the proper value for the SC provider for now.

The SC that was automatically created came from Juju. It seems that Juju will need to detect whether the CSI plugin is available. Additionally, there was a recent upstream change[1] to the CSI plugin which affects the provider value. So Juju will also need to check the version of the plugin to determine which provider value to use for the SC:

* no CSI plugin: kubernetes.io/cinder
* CSI plugin less than v1.2.0: csi-cinderplugin
* CSI plugin v1.2.0 or greater: cinder.csi.openstack.org

[1]: https://github.com/kubernetes/cloud-provider-openstack/pull/624