Juju agents cannot start: failed to start "uniter" manifold worker: dependency not available

[Jorge Niedbalski]

[Environment]

Xenial
Juju-Core beta 7 (Stuck in beta 7 because no upgrade paths are available)

[Description]

Restarting jujud for the unit, the unit
remains in failed/error state, the following logs are printed (enabled TRACE level):

2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:432 registered "leadership-tracker" manifold worker
2016-08-23 18:21:01 DEBUG juju.worker.dependency engine.go:465 "leadership-tracker" manifold worker started
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:587 restarting dependents of "leadership-tracker" manifold
2016-08-23 18:21:01 DEBUG juju.worker.leadership tracker.go:122 ceph-osd/10 making initial claim for ceph-osd leadership
2016-08-23 18:21:01 DEBUG juju.worker.leadership tracker.go:164 checking ceph-osd/10 for ceph-osd leadership
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:401 starting "uniter" manifold worker in 10ms...
2016-08-23 18:21:01 DEBUG juju.worker.dependency engine.go:479 "leadership-tracker" manifold worker stopped: leadership failure: lease manager stopped
2016-08-23 18:21:01 ERROR juju.worker.dependency engine.go:526 "leadership-tracker" manifold worker returned unexpected error: leadership failure: lease manager stopped
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:587 restarting dependents of "leadership-tracker" manifold
2016-08-23 18:21:01 DEBUG juju.worker.dependency engine.go:479 "uniter" manifold worker stopped: <nil>
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:401 starting "leadership-tracker" manifold worker in 3s...
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:401 starting "uniter" manifold worker in 10ms...
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:410 starting "uniter" manifold worker
2016-08-23 18:21:01 TRACE juju.worker.dependency context.go:44 "uniter" manifold requested "agent" resource
2016-08-23 18:21:01 TRACE juju.worker.dependency context.go:44 "uniter" manifold requested "api-caller" resource
2016-08-23 18:21:01 TRACE juju.worker.dependency context.go:44 "uniter" manifold requested "machine-lock" resource
2016-08-23 18:21:01 TRACE juju.worker.dependency context.go:44 "uniter" manifold requested "leadership-tracker" resource
2016-08-23 18:21:01 TRACE juju.worker.dependency engine.go:422 failed to start "uniter" manifold worker: dependency not available
2016-08-23 18:21:01 DEBUG juju.worker.dependency engine.go:479 "uniter" manifold worker stopped: dependency not available

[Anastasia (anastasia-macmood)]

Jorge will provide more information including useful context - what provider is used, setup description like how many units, what units are ok or failing, logs, etc.

[Anastasia (anastasia-macmood)]

As discussed offline, please stop all controllers and run MgoPurge from https://github.com/juju/juju/wiki/Incomplete-Transactions-and-MgoPurge. This should fix the database and once controllers are restarted, you'll be in a better place \o/

[Launchpad Janitor (janitor)]

[Expired for juju because there has been no activity for 60 days.]