The 'next' keystone charm has an always-on haproxy configuration which determine_api_port() is aware of. When the upgrade-charm hook fires get_local_endpoint() uses determine_api_port() to return the endpoint to enable direct communication with keystone on the lower admin port 35347. However, CONFIGS.write_all() hasn't run yet so keystone is still listening on 35357. This results in the hook failing with:
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 Traceback (most recent call last):
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/upgrade-charm", line 427, in <module>
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 main()
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/upgrade-charm", line 421, in main
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 hooks.execute(sys.argv)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/charmhelpers/core/hookenv.py", line 528, in execute
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 self._hooks[hook_name]()
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/charmhelpers/core/host.py", line 296, in wrapped_f
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 f(*args)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/upgrade-charm", line 398, in upgrade_charm
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 ensure_initial_admin(config)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/keystone_utils.py", line 539, in ensure_initial_admin
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 create_tenant("admin")
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/var/lib/juju/agents/unit-keystone-0/charm/hooks/keystone_utils.py", line 415, in create_tenant
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 tenants = [t._info for t in manager.api.tenants.list()]
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/v2_0/tenants.py", line 118, in list
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 tenant_list = self._list("/tenants%s" % query, "tenants")
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/base.py", line 106, in _list
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 resp, body = self.client.get(url)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 590, in get
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 return self._cs_request(url, 'GET', **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 582, in _cs_request
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 return self.request(url, method, **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py", line 564, in request
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 resp = super(HTTPClient, self).request(url, method, **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/baseclient.py", line 21, in request
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 return self.session.request(url, method, **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/utils.py", line 318, in inner
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 return func(*args, **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 242, in request
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 resp = self._send_request(url, method, redirect, **kwargs)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 File "/usr/lib/python2.7/dist-packages/keystoneclient/session.py", line 271, in _send_request
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 raise exceptions.ConnectionError(msg)
2015-01-21 15:50:49 INFO unit.keystone/0.upgrade-charm logger.go:40 keystoneclient.exceptions.ConnectionError: Unable to establish connection to http://localhost:35347/v2.0/tenants
2015-01-21 15:50:49 ERROR juju.worker.uniter uniter.go:608 hook "upgrade-charm" failed: exit status 1
