Default storage class on bundle deploy

Bug #1815967 reported by Ian Booth
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Kubernetes Control Plane Charm
Fix Released
Medium
Mike Wilson

Bug Description

I couldn't see the best project to file this under.
When CDK is deployed, we need to have a cluster default storage class created which provisions relevant disks for the underlying cloud, eg
google - gce-pd
aws - ebs
azure - azure disk

This is needed to allow Juju to work seamlessly on supported clouds.

Revision history for this message
Cory Johns (johnsca) wrote :

We had some discussion about this today. We could certainly add logic in the k8s charms to create an appropriate SC out of the box depending on which, if any, integrator charm is available. However, due to the differing levels of support from the various providers and the need to handle the case where an integrator charm isn't available, it may be that another approach would give us a more consistent and reliable level of support.

Specifically, it seems that CDK + CephFS is already being deployed in the field, and the k8s charms have support for CephFS and the CephFS CSI plugin. If we made this part of the default CDK bundle, and preferentially use that to back the standard storage class, it would be a much nicer experience out of the box. Additionally, if we're smart about placement, we could do this without adding any additional machines to the bundle.

The main caveats are:

* CephFS doesn't work under LXD, due to the need for block device access for storage (at least according to my current understanding). This would limit deploying CDK on a LXD provider.

* Juju would still need to be able to handle being handed a non-CDK cluster that may not provide the standard SC. But this would apply equally to the integrator approach and eventually, we can't get around the limitation that we can't introspect what provisioners are available on a random cluster, so we have to rely on the cluster admin (or charms) to provide a SC we can use.

Revision history for this message
Cory Johns (johnsca) wrote :

Actually, I'm not certain that LXD is a limitation for CephFS, since you can deploy it into Kubernetes: http://docs.ceph.com/docs/mimic/start/kube-helm/

That could actually be another approach, which I think we discussed previously; instead of relying on the cluster to provide a reliable storage provider, just require some random default storage class and when Juju creates a model (or lazily), deploy CephFS into the cluster and then use that for whatever storage Juju needs. We get the guarantee of full feature support, while keeping the pre-requirements to a minimum.

Revision history for this message
Ian Booth (wallyworld) wrote :

The requirement is to provide out of the box a storage class which provisions opinionated default disk volumes from the underlying cloud. Such disks are mounted into the pods as required to satisfy the standard Juju storage semantics. Cephfs out of the box may be a good thing to do as well, but it's not relevant in this case.

Revision history for this message
Cory Johns (johnsca) wrote :

Sorry, the bit about in-cluster Ceph was me getting side-tracked on a different problem, unrelated to this bug.

However, my original point was that I don't think that we should bother with making CDK back its default storage class with cloud-backed storage, which would require the CDK charms to know the proper provider definition for each underlying cloud as well as a fallback when no cloud integration was available. Instead, I think it would be better, more in line with field best practices and lessons learned from OpenStack, and easier to maintain in the CDK charms to just include Ceph with CDK and always back the default storage class with that, regardless of what the underlying cloud is.

Revision history for this message
Cory Johns (johnsca) wrote :

Although, if we need to ensure that k8s-master *always* configures a default storage class, regardless of how it's deployed, then I feel like the order of preference should be:

* CephFS
* Cloud integration
* HostPath (backed by Juju Storage)

Revision history for this message
George Kraft (cynerva) wrote :

To be clear, CDK does not support CephFS today. The provisioner that we use on Ceph is RBD. I'm pretty sure field does the same.

Revision history for this message
Mike Wilson (knobby) wrote :

PR to add the default storage classes for AWS, Google Cloud, and Azure:

https://github.com/charmed-kubernetes/charm-kubernetes-master/pull/71/files

Changed in charm-kubernetes-master:
assignee: nobody → Mike Wilson (knobby)
importance: Undecided → Medium
milestone: none → 1.17
Mike Wilson (knobby)
Changed in charm-kubernetes-master:
status: New → In Progress
Changed in charm-kubernetes-master:
status: In Progress → Fix Committed
Changed in charm-kubernetes-master:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.