"juju destroy-controller" can leak hosted models

[Andrew Wilkins]

juju destroy-controller requires you to first destroy hosted models. So I ran this after a fresh bootstrap:

    juju add-machine
    (wait for machine to be provisioned)
    juju destroy-model -y default && juju destroy-controller -y ctrl

And the hosted model machine is still there.

The code in state that decrements hosted-model refcounts is broken. When you create a hosted model, a refcount is added to the controller. As long as this refcount is non-zero, you cannot destroy the controller. When you "juju destroy-model", it marks the model as dying, enqueues cleanups (destroy services, machines, etc.), and decrements the refcount. It's then possible to destroy the controller before all of the hosted model resources are cleaned up.

[Andrew Wilkins (axwalk)]

I'm tempted to target for 2.0, but I don't think it'll get fixed in time. This is complicated, due to how we keep models hanging around even after they're dead. Normally we would decrement refcounts when removing entities from state. I suspect we'll just have to do it when setting it to Dead, rather than Dying.

[Andrew Wilkins (axwalk)]

Actually, that shouldn't be too hard. Working on it now.

[Andrew Wilkins (axwalk)]

https://github.com/juju/juju/pull/5048