Comment 35 for bug 1818113

Revision history for this message
Aurelien Lourot (aurelien-lourot) wrote :

debug_log.txt contains lines like:

unit-keystone-2: 19:59:30 DEBUG unit.keystone/2.juju-log cluster:30: Telling peers this unit is: NOTREADY
unit-keystone-2: 20:12:50 INFO unit.keystone/2.juju-log Keystone charm unit not ready - deferring identity-relation updates

but never contains lines like:

DEBUG unit.keystone/?.juju-log Telling peers this unit is: READY
DEBUG unit.keystone/?.juju-log Application Ready
INFO unit.keystone/?.juju-log Firing identity_credentials_changed hook for all related services.

Based on the source code this proves that the keystone charm has correctly assessed that at least one unit wasn't ready and thus didn't mark the application as ready and also didn't update the identity-* relations. Debugging the charm locally shows that these things correctly happen later eventually.

However there is a special code-path [0] which sets 'service_password' on the identity-service relation with no regard for peer-readiness whatsoever. This path finally prints this line, which has been seen early in the logs:

unit-keystone-1: 19:57:06 INFO unit.keystone/1.juju-log identity-service:115: Deferring identity_changed() to service leader.

I don't have a precise solution yet but I'm now pretty sure this is the root cause.

0: https://github.com/openstack/charm-keystone/blob/master/hooks/keystone_hooks.py#L486