leader-elected fails if leadership was lost in the meantime, breaking sync

Bug #1715784 reported by Paul Collins
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
ubuntu-repository-cache (Juju Charms Collection)
Confirmed
Undecided
Unassigned

Bug Description

If a unit is deposed with the leadership-elected hook queued, and leadership-elected subsequently runs, it fails messily and blocks syncs, e.g.:

2017-09-07 20:52:43 ERROR juju.worker.dependency engine.go:547 "api-caller" manifold worker returned unexpected error: cannot open api: try again (try again)
2017-09-07 20:53:01 WARNING juju.worker.uniter.operation leader.go:115 we should run a leader-deposed hook here, but we can't yet
2017-09-07 20:53:02 INFO leader-elected error: cannot write leadership settings: cannot write settings: not the leader
2017-09-07 20:53:02 INFO leader-elected Traceback (most recent call last):
2017-09-07 20:53:02 INFO leader-elected File "/var/lib/juju/agents/unit-ubuntu-repository-cache-0/charm/hooks/leader-elected", line 245, in <module>
2017-09-07 20:53:02 INFO leader-elected HOOKS.execute(sys.argv)
2017-09-07 20:53:02 INFO leader-elected File "/var/lib/juju/agents/unit-ubuntu-repository-cache-0/charm/lib/charmhelpers/core/hookenv.py", line 715, in execute
2017-09-07 20:53:02 INFO leader-elected self._hooks[hook_name]()
2017-09-07 20:53:02 INFO leader-elected File "/var/lib/juju/agents/unit-ubuntu-repository-cache-0/charm/hooks/leader-elected", line 226, in leader_elected
2017-09-07 20:53:02 INFO leader-elected hookenv.leader_set(leader_id=hookenv.local_unit())
2017-09-07 20:53:02 INFO leader-elected File "/var/lib/juju/agents/unit-ubuntu-repository-cache-0/charm/lib/charmhelpers/core/hookenv.py", line 837, in inner_translate_exc2
2017-09-07 20:53:02 INFO leader-elected return f(*args, **kwargs)
2017-09-07 20:53:02 INFO leader-elected File "/var/lib/juju/agents/unit-ubuntu-repository-cache-0/charm/lib/charmhelpers/core/hookenv.py", line 890, in leader_set
2017-09-07 20:53:02 INFO leader-elected subprocess.check_call(cmd)
2017-09-07 20:53:02 INFO leader-elected File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
2017-09-07 20:53:02 INFO leader-elected raise CalledProcessError(retcode, cmd)
2017-09-07 20:53:02 INFO leader-elected subprocess.CalledProcessError: Command '['leader-set', 'leader_id=ubuntu-repository-cache/0']' returned non-zero exit status 1
2017-09-07 20:53:02 ERROR juju.worker.uniter.operation runhook.go:107 hook "leader-elected" failed: exit status 1

This can be difficult to notice because by default Juju 2 will regularly retry the hook. Running "juju resolve --no-retry ubuntu-repository-cache/0" skips the hook and lets the sync complete properly.

Haw Loeung (hloeung)
Changed in ubuntu-repository-cache (Juju Charms Collection):
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.