Model is empty and not empty at the same time

[Leon]

I destroy-model & destroy-storage a model, but it refused to go away.

$ juju models:

Controller: ctl1

Model Cloud/Region Type Status Units Access Last connection
b1* microk8s/localhost kubernetes destroying - admin 2 seconds ago
b2 microk8s/localhost kubernetes available 5 admin 7 minutes ago
controller microk8s/localhost kubernetes available 1 admin just now

Then I added --force --no-wait, and it still hangs.

$ juju status:

- model notes say "model not empty"
- status body says Model "admin/b1" is empty
- juju kill-

Model Controller Cloud/Region Version SLA Timestamp Notes
b1 ctl1 microk8s/localhost 3.2.2 unsupported 13:00:57-04:00 attempt 1 to destroy model failed (will retry): model not empty, found 4 applications, 3 volumes, 3 filesystems (mod...

Model "admin/b1" is empty.

------

$ microk8s.kubectl get services -A | grep trfk:

- The old model (b1) still lists traefik, and for this reason the new model (b2) can't get an external IP (says "pending"; I have only one IP in metallb addresspool).

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
b1 trfk-endpoints ClusterIP None <none> <none> 38m
b1 trfk LoadBalancer 10.152.183.244 10.43.8.149 80:32481/TCP,443:31709/TCP 38m
b2 trfk-endpoints ClusterIP None <none> <none> 19m
b2 trfk LoadBalancer 10.152.183.149 <pending> 80:31093/TCP,443:31729/TCP 19m

[Leon (sed-i)]

Forgot to mention that this happened after ~5 min after I attempted `upgrade-controller` from 3.2.0 to 3.2.2, which seemed to not work.

[Leon (sed-i)]

Seems like it's a 3.2.2 bug: I get the same problem repeatedly when destroying models in the same 3.2.2 controller.

[Harry Pidcock (hpidcock)]

To do a truly forceful remove you now need to pass `--force --no-wait --timeout 0s`, and there is a bug in 3.2.2 and 3.2.3 where you may need to restart the controller for it to notice you changed the parameters of model destruction. This should be fixed in 3.2.4, when we release it.

[Harry Pidcock (hpidcock)]

As to why the model is stuck destroying, we would need juju controller and model logs, kubectl describe output for pods/statefulsets/namespace.

[Launchpad Janitor (janitor)]

[Expired for Canonical Juju because there has been no activity for 60 days.]