error tracker does not work with django 1.5

[Brian Murray]

The ubuntu error tracker is currently configured to use python-django version 1.3 and fails to run with version 1.5.4 (available in the ubuntu-cloud archive). There are multiple issues:

This traceback (https://pastebin.canonical.com/106166/) occurs when trying to run:

PYTHONPATH=/srv/errors.staging.ubuntu.com/staging/local_config:/srv/errors.staging.ubuntu.com/staging/daisy:/srv/errors.staging.ubuntu.com/staging/errors /srv/errors.staging.ubuntu.com/staging/errors/tools/add_teams.py

This is resolvable by setting the following environmental variable:

DJANGO_SETTINGS_MODULE=errors.settings

However, then when trying to run python-django 1.5 the following traceback appears;

File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 92, in get_response
    response = middleware_method(request)
  File "/usr/lib/python2.7/dist-packages/django/middleware/common.py", line 57, in process_request
    host = request.get_host()
  File "/usr/lib/python2.7/dist-packages/django/http/request.py", line 72, in get_host
    "Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): %s" % host)

That was resolvable by setting ALLOWED_HOSTS (in settings.py) to ['errors.staging.ubuntu.com']

After that there was another traceback:

E__init__() got an unexpected keyword argument 'namedtuple_as_object'

File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/dist-packages/django/views/decorators/csrf.py", line 77, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 264, in wrapper
    return self._handle_500(request, e)
  File "/srv/errors.staging.ubuntu.com/production/errors/errors/api/resources.py", line 88, in _handle_500
    return Resource._handle_500(self, request, exception)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 315, in _handle_500
    return self.error_response(request, data, response_class=response_class)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 1243, in error_response
    serialized = self.serialize(request, errors, desired_format)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 397, in serialize
    return self._meta.serializer.serialize(data, format, options)
  File "/usr/lib/python2.7/dist-packages/tastypie/serializers.py", line 187, in serialize
    serialized = getattr(self, "to_%s" % desired_format)(bundle, options)
  File "/srv/errors.staging.ubuntu.com/production/errors/errors/api/resources.py", line 56, in to_json
    sort_keys=True, ensure_ascii=False, indent=self.json_indent)
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 296, in dumps
    **kw).encode(obj)

I'm not certain how to sort that one out.

[Brian Murray (brian-murray)]

Oh, and python-tastypie had to be upgrade to version 0.9.15-1 because of the movement of LOOKUPL_SEP - see tastypie/resources.py:

# Django 1.5 has moved this constant up one level.
try:
    from django.db.models.constants import LOOKUP_SEP
except ImportError:
    from django.db.models.sql.constants import LOOKUP_SEP

[Evan (ev)]

Could we not apt pin django and tastypie to the version we know to work?

[Brian Murray (brian-murray)]

Yes, but I don't think that's a good long term solution.

[Brian Murray (brian-murray)]

This might be the source of the problem:

https://docs.djangoproject.com/en/1.5/releases/1.5/#system-version-of-simplejson-no-longer-used

[Brian Murray (brian-murray)]

This in particular has detail about how to potentially fix it.

https://docs.djangoproject.com/en/1.5/releases/1.5/#simplejson-deprecation

[Brian Murray (brian-murray)]

 $ bzr cdiff
=== modified file 'errors/api/resources.py'
--- errors/api/resources.py 2014-04-28 20:28:56 +0000
+++ errors/api/resources.py 2014-05-29 14:56:58 +0000
@@ -8,13 +8,13 @@
 TASTYPIE_FULL_DEBUG = True

 from django.core.serializers import json
-from django.utils import simplejson
 from tastypie.serializers import Serializer
 from ..metrics import measure_view
 from daisy import launchpad
 from operator import itemgetter
 import apt
 import datetime
+import simplejson
 from hashlib import sha1
 from urllib import quote
 from urllib import unquote

[Brian Murray (brian-murray)]

Comment #6 seems to have worked but I received yet another traceback.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 217, in wrapper
    response = callback(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 459, in dispatch_list
    return self.dispatch('list', request, **kwargs)
  File "/var/www/errors/errors/metrics.py", line 40, in wrapper
    result = view(*args, **kwargs)
  File "/var/www/errors/errors/api/resources.py", line 86, in dispatch
    return Resource.dispatch(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 491, in dispatch
    response = method(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 1299, in get_list
    objects = self.obj_get_list(bundle=base_bundle, **self.remove_api_resource_names(kwargs))
TypeError: obj_get_list() got an unexpected keyword argument 'bundle'

[Brian Murray (brian-murray)]

I actually used 'import json as simplejson' to fix comment #6.

[Brian Murray (brian-murray)]

Removing the statsd timer wrapper had little change on the traceback.

 class ErrorsResource(Resource):
     # Wrap the view dispatch with a statsd timing.
- @measure_view
- def dispatch(self, *args, **kwargs):
- return Resource.dispatch(self, *args, **kwargs)
+ #@measure_view
+ #def dispatch(self, *args, **kwargs):
+ # return Resource.dispatch(self, *args, **kwargs)

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 217, in wrapper
    response = callback(request, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 459, in dispatch_list
    return self.dispatch('list', request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 491, in dispatch
    response = method(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 1299, in get_list
    objects = self.obj_get_list(bundle=base_bundle, **self.remove_api_resource_names(kwargs))
TypeError: obj_get_list() got an unexpected keyword argument 'bundle'

[Evan (ev)]

  File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 1299, in get_list
    objects = self.obj_get_list(bundle=base_bundle, **self.remove_api_resource_names(kwargs))
TypeError: obj_get_list() got an unexpected keyword argument 'bundle'

I suspect we just need to update to match the new tastypie API.

[Brian Murray (brian-murray)]

While that's true there seems to be something wrong with tastypie:

File "/usr/lib/python2.7/dist-packages/tastypie/resources.py", line 1299, in get_list
    objects = self.obj_get_list(bundle=base_bundle, **self.remove_api_resource_names(kwargs))

and obj_get_list:

   def obj_get_list(self, bundle, **kwargs):
        """
        A ORM-specific implementation of ``obj_get_list``.

        Takes an optional ``request`` object, whose ``GET`` dictionary can be
        used to narrow the query.
        """

[Brian Murray (brian-murray)]

Ah, no it seems to have been an issue with what we were passing to obj_get_list.