Update ceph-csi to 3.1.0 to enable CephFS snapshots

[David van der Spek]

I am busy creating a CDK cluster for Kubeflow, and one of the main selling points for our use is to have Kale create pipelines from notebook servers. At the moment Kale only support Rok for storage, and requires ReadWriteMany volumes, snapshotting and creating PVC's from snapshots for full functionality. The Ceph CSI release 2.1.2 that is used by CDK does not allow the creation of CephFS snapshots. However, Ceph CSI release 3.1.0 adds all these functionalities.

[Chris Sanders (chris.sanders)]

This wasn't completed before the 1.20 work, I'm going to add the 1.20+ck1 milestone here to indicate that we will see if it can be made available in a point release. This shouldn't be taken as a commitment that it will be available in 1.20+ck1 if the changes between the 2.x and 3.x CSI are significant it might h ave to be targeted at a later release.

[Kevin W Monroe (kwmonroe)]

PR for review to go to ceph-csi v3.2.1:

https://github.com/charmed-kubernetes/cdk-addons/pull/204

[Kevin W Monroe (kwmonroe)]

Unfortunately, we can't upgrade to 3.x at the moment. In 3.x, PVCs created with 1.x are no longer supported. We used 1.x as recently as CK 1.19, so we'll need a clear upgrade path for users that may have 1.x PVCs before we can fix this.

[David van der Spek (vanderspek-david)]

Wouldn't it be an option to expose a setting to allow the user to choose what version of the CSI driver to use? Seeing as snapshots are now GA, this is a major functionality that is missing with the Ceph charms and CDK.

[Kevin W Monroe (kwmonroe)]

We needed to go to v3.3.1 for the ck-1.22 release due to the removal of v1beta1 k8s api support.

PR:

https://github.com/charmed-kubernetes/cdk-addons/pull/204