KeyError: 'tenant_usage' base.py line 151 in _get

[Jason Brittain]

I have a new installation of the Dashboard. Dashboard logins fail to complete, and this is what I find in /var/log/apache2/error.log:

[Wed Mar 28 11:14:52 2012] [warn] mod_wsgi: Compiled for Python/2.7.2rc1.
[Wed Mar 28 11:14:52 2012] [warn] mod_wsgi: Runtime using Python/2.7.2+.
[Wed Mar 28 11:14:52 2012] [notice] Apache/2.2.20 (Ubuntu) mod_wsgi/3.3 Python/2.7.2+ configured -- resuming normal operations
[Wed Mar 28 11:15:00 2012] [error] DEBUG:openstack_dashboard.settings:Running in debug mode without debug_toolbar.
[Wed Mar 28 11:15:01 2012] [error] DEBUG:openstack_dashboard.settings:Running in debug mode without debug_toolbar.
[Wed Mar 28 11:15:05 2012] [error] CRITICAL:openstack_dashboard:Unhandled Exception in of type "<type 'exceptions.KeyError'>" in dashboard.
[Wed Mar 28 11:15:05 2012] [error] Traceback (most recent call last):
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/pymodules/python2.7/django/core/handlers/base.py", line 111, in get_response
[Wed Mar 28 11:15:05 2012] [error] response = callback(request, *callback_args, **callback_kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 40, in dec
[Wed Mar 28 11:15:05 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 57, in dec
[Wed Mar 28 11:15:05 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 40, in dec
[Wed Mar 28 11:15:05 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/pymodules/python2.7/django/views/generic/base.py", line 47, in view
[Wed Mar 28 11:15:05 2012] [error] return self.dispatch(request, *args, **kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/pymodules/python2.7/django/views/generic/base.py", line 68, in dispatch
[Wed Mar 28 11:15:05 2012] [error] return handler(request, *args, **kwargs)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 100, in get
[Wed Mar 28 11:15:05 2012] [error] handled = self.construct_tables()
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 87, in construct_tables
[Wed Mar 28 11:15:05 2012] [error] data = self._get_data_dict()
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 127, in _get_data_dict
[Wed Mar 28 11:15:05 2012] [error] self._data = {self.table_class._meta.name: self.get_data()}
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/dashboards/nova/overview/views.py", line 30, in get_data
[Wed Mar 28 11:15:05 2012] [error] super(ProjectOverview, self).get_data()
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/usage/views.py", line 33, in get_data
[Wed Mar 28 11:15:05 2012] [error] self.usage.summarize(*self.usage.get_date_range())
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/usage/base.py", line 94, in summarize
[Wed Mar 28 11:15:05 2012] [error] _('Unable to retrieve usage information.'))
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/usage/base.py", line 91, in summarize
[Wed Mar 28 11:15:05 2012] [error] self.usage_list = self.get_usage_list(start, end)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/usage/base.py", line 126, in get_usage_list
[Wed Mar 28 11:15:05 2012] [error] usage = api.usage_get(self.request, self.tenant_id, start, end)
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/api/nova.py", line 394, in usage_get
[Wed Mar 28 11:15:05 2012] [error] return Usage(novaclient(request).usage.get(tenant_id, start, end))
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/novaclient/v1_1/usage.py", line 48, in get
[Wed Mar 28 11:15:05 2012] [error] "tenant_usage")
[Wed Mar 28 11:15:05 2012] [error] File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 151, in _get
[Wed Mar 28 11:15:05 2012] [error] return self.resource_class(self, body[response_key])
[Wed Mar 28 11:15:05 2012] [error] KeyError: 'tenant_usage'

I see others reporting the same problem. In the latest source of python-novaclient, the relevant code has been refactored since essex rc1.

[Gabriel Hurley (gabriel-hurley)]

Can you shed some light on how you installed Horizon and its dependencies? Was this from devstack, debian packages, manual installation?

[Mike Perez (thingee)]

I've just spun up a new instance of the latest with devstack and couldn't reproduce this when trying to login to the dashboard.

[Gabriel Hurley (gabriel-hurley)]

Yeah, I'm certain this isn't a devstack issue, and doesn't happen with github master for horizon and python-novaclient. That's why I asked about the install method. It's almost certainly a dependency version problem, but I'd like to know which sources were used to see if there's anything we *can* do about it.

[Jason Brittain (jason-brittain)]

I installed all of openstack on Ubuntu 11.04 using the milestone-proposed (rc1) deb packages from here:

https://launchpad.net/~openstack-ppa/+archive/milestone-proposed

I can give you any information you want about what versions of things are installed, and run commands you want run to get info about the installation. Just let me know.

[Jason Brittain (jason-brittain)]

I have attached a patch that avoids the bug, changing usage.py in novaclient. I'm not sure if this is the right fix, but after this change I can log into the console. If I then click on any of the tabs, I get errors. Example:

[Wed Mar 28 22:03:12 2012] [error] CRITICAL:openstack_dashboard:Unhandled Exception in of type "<type 'exceptions.KeyError'>" in dashboard.
[Wed Mar 28 22:03:12 2012] [error] Traceback (most recent call last):
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/pymodules/python2.7/django/core/handlers/base.py", line 111, in get_response
[Wed Mar 28 22:03:12 2012] [error] response = callback(request, *callback_args, **callback_kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 40, in dec
[Wed Mar 28 22:03:12 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 55, in dec
[Wed Mar 28 22:03:12 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/decorators.py", line 40, in dec
[Wed Mar 28 22:03:12 2012] [error] return view_func(request, *args, **kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/pymodules/python2.7/django/views/generic/base.py", line 47, in view
[Wed Mar 28 22:03:12 2012] [error] return self.dispatch(request, *args, **kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/pymodules/python2.7/django/views/generic/base.py", line 68, in dispatch
[Wed Mar 28 22:03:12 2012] [error] return handler(request, *args, **kwargs)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 105, in get
[Wed Mar 28 22:03:12 2012] [error] handled = self.construct_tables()
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 96, in construct_tables
[Wed Mar 28 22:03:12 2012] [error] handled = self.handle_table(table)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 68, in handle_table
[Wed Mar 28 22:03:12 2012] [error] data = self._get_data_dict()
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/tables/views.py", line 37, in _get_data_dict
[Wed Mar 28 22:03:12 2012] [error] self._data[table._meta.name] = data_func()
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/dashboards/nova/instances_and_volumes/views.py", line 52, in get_instances_data
[Wed Mar 28 22:03:12 2012] [error] exceptions.handle(self.request, _('Unable to retrieve instances.'))
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/dashboards/nova/instances_and_volumes/views.py", line 49, in get_instances_data
[Wed Mar 28 22:03:12 2012] [error] instances = api.server_list(self.request)
[Wed Mar 28 22:03:12 2012] [error] File "/usr/lib/python2.7/dist-packages/horizon/api/nova.py", line 309, in server_list
[Wed Mar 28 22:03:12 2012] [error] servers.list(True, search_opts)]
[Wed Mar 2...

[Dolph Mathews (dolph)]

This turned out to be the result of an invalid endpoint configuration for nova in keystone.

Specifically, the endpoint was configured as:

    http://10.254.99.99:8774/v2.0

... instead of:

    http://10.254.99.99:8774/v2/$(tenant_id)s