Comment 7 for bug 1721786

Revision history for this message
Anastasia (anastasia-macmood) wrote :

After further investigation, I can see that we definitely can end up in a twist.

It looks like we may not release a connection to a model in a timely manner after the model has been destroyed (pool item flagged as removed).

I have seen similar errors from other API calls and in the workers' code. It can actually be fairly frequent, depending on the model and how saturated it was - deployed charms, users, etc.
This is just a small sample of places that I have extracted both from the log output above as well as from my own testing:

* "model-upgrader" [DEBUG juju.worker.dependency engine.go:553 stack trace:
model XXX has been removed
github.com/juju/juju/worker/modelupgrader/worker.go:86:
github.com/juju/juju/worker/modelupgrader/manifold.go:66:]

* Controller[""].AllModels

* ModelManager[""].ModelStatus

* ModelManager[""].ListModels

I will work on tracing where we are not releasing model reference.
I will also investigate all callers of the pool.Get() and pool.GetModel() to consider how they should handle NotFound error.