leftover ReplicaSets when using scalePolicy: serial on k8s charm

[Evan Hanson]

When upgrading a charm with files and a scalePolicy of "serial" in a way
that causes a new unit to be started, Juju will leave behind an unused
ReplicaSet with replicas: 0.

        $ cat config/podspec.yaml
        version: 2
        service:
          scalePolicy: serial
        containers:
          - name: foo
            imageDetails:
              imagePath: docker.io/library/nginx
            ports:
              - containerPort: 80
                protocol: TCP
            files:
              - name: foo
                mountPath: /foo
                files:
                  foo: bar

        $ juju deploy .
        Deploying charm "local:kubernetes/traefik-0".

        $ kubectl -n traefik get all
        NAME READY STATUS RESTARTS AGE
        pod/traefik-64bdf568b6-5mrgk 1/1 Running 0 24s
        pod/traefik-operator-0 1/1 Running 0 45s

        NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
        service/traefik ClusterIP 10.254.118.4 <none> 80/TCP 24s
        service/traefik-operator ClusterIP 10.254.82.155 <none> 30666/TCP 45s

        NAME READY UP-TO-DATE AVAILABLE AGE
        deployment.apps/traefik 1/1 1 1 24s

        NAME DESIRED CURRENT READY AGE
        replicaset.apps/traefik-64bdf568b6 1 1 1 24s

        NAME READY AGE
        statefulset.apps/traefik-operator 1/1 45s

If you make a change to the podspec so that a new unit is deployed (I
just changed the name and mountPath of the file, which seems to work
reliably), then uprade the charm and wait a moment, you get one new
ReplicaSet and one left over. Note this doesn't occur if the worker is
updated in-place (i.e. if it doesn't start a new unit), I guess because
in that case the scalePolicy doesn't come into play.

        $ juju upgrade-charm --path . traefik
        Added charm "local:kubernetes/traefik-1" to the model.

        $ kubectl -n traefik get rs
        NAME DESIRED CURRENT READY AGE
        replicaset.apps/traefik-7db48d7cf 0 0 0 3m7s
        replicaset.apps/traefik-8d768dd89 1 1 1 45s

Do this a few more times and you get a new one for each unit:

        $ kubectl -n traefik get rs
        NAME DESIRED CURRENT READY AGE
        replicaset.apps/traefik-5d448b7467 1 1 1 37s
        replicaset.apps/traefik-64bdf568b6 0 0 0 4m12s
        replicaset.apps/traefik-748ff464b7 0 0 0 116s
        replicaset.apps/traefik-7db48d7cf 0 0 0 9m21s
        replicaset.apps/traefik-8d768dd89 0 0 0 6m59s

These old replicasets have replicas set to "0" (rather than being
deleted as we would expect).

        $ kubectl -n traefik get replicaset.apps/traefik-8d768dd89 -o yaml | paste.ubuntu.com
        # https://paste.ubuntu.com/p/XwdgQc9YTV/

Thanks for looking, and sorry for the delay getting these filed.

Evan

[Yang Kelvin Liu (kelvin.liu)]

https://github.com/juju/juju/pull/11269 will land to 2.8 to fix it;