auth_token causes services to fail to start if keystone is down

[Dan Prince]

Using python-keystoneclient git d782a998 on Fedora.

Services will now fail to startup correctly if the Keystone server is not running. The following example stack trace from glance shows the issue:

2013-03-13 15:09:01.020 23171 CRITICAL glance [-] Unable to communicate with keystone
2013-03-13 15:09:01.020 23171 TRACE glance Traceback (most recent call last):
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/bin/glance-api", line 60, in <module>
2013-03-13 15:09:01.020 23171 TRACE glance server.start(config.load_paste_app(), default_port=9292)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/glance/common/config.py", line 205, in load_paste_app
2013-03-13 15:09:01.020 23171 TRACE glance app = deploy.loadapp("config:%s" % conf_file, name=app_name)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2013-03-13 15:09:01.020 23171 TRACE glance return loadobj(APP, uri, name=name, **kw)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2013-03-13 15:09:01.020 23171 TRACE glance return context.create()
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2013-03-13 15:09:01.020 23171 TRACE glance return self.object_type.invoke(self)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 207, in invoke
2013-03-13 15:09:01.020 23171 TRACE glance app = filter(app)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 1163, in auth_filter
2013-03-13 15:09:01.020 23171 TRACE glance return AuthProtocol(app, conf)
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 338, in __init__
2013-03-13 15:09:01.020 23171 TRACE glance self.auth_version = self._choose_api_version()
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 392, in _choose_api_version
2013-03-13 15:09:01.020 23171 TRACE glance versions_supported_by_server = self._get_supported_versions()
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 412, in _get_supported_versions
2013-03-13 15:09:01.020 23171 TRACE glance response, data = self._json_request('GET', '/')
2013-03-13 15:09:01.020 23171 TRACE glance File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 618, in _json_request
2013-03-13 15:09:01.020 23171 TRACE glance raise ServiceError('Unable to communicate with keystone')

-----

To reproduce:

1) stop keystone

2) restart glance-api

Expected behaviour:

glance-api should startup regardless of of whether keystone is running.

Observed behaviour:

glance-api fails to startup.

-----

Fix: The easiest way to fix this is to adjust the latest middleware so that it using lazy loading to set the auth_version used.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24370

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24370
Committed: http://github.com/openstack/python-keystoneclient/commit/db7603824ff541b4319bb193d5861ac6af570b37
Submitter: Jenkins
Branch: master

commit db7603824ff541b4319bb193d5861ac6af570b37
Author: Dan Prince <email address hidden>
Date: Wed Mar 13 17:33:22 2013 -0400

    Make auth_token lazy load the auth_version.

    Updates recent changes to the auth_token middleware (this
    is a regression in d782a99) so that self.auth_version is
    lazy loaded.

    This fixes issues where other openstack services would fail
    to startup correctly if Keystone is not running. The issue
    was auth_token was trying to make a request to '/' to get
    versions information on startup to "autodetect" the correct
    version to use.

    This patch fixes startup issues by moving the version detection
    so that it is lazy loaded right before it is actually used.

    This issue should fix SmokeStack :)

    Fixes LP Bug #1154806.

    Change-Id: Ib24f5386fa1ffe0e0365548840f0cfeaae36f548