HTTP 500 Internal server error when calling scheduler.pending_jobs_by_device_type over XML-RPC to v.l.o

[Matthew Hart]

This chunk of code produces an HTTP 500 from v.l.o:

import xmlrpclib
server = xmlrpclib.ServerProxy("http://validation.linaro.org/RPC2")
print server.scheduler.pending_jobs_by_device_type()

xmlrpclib.ProtocolError: <ProtocolError for validation.linaro.org/RPC2: 500 Internal Server Error>

However change the url to staging or playground and you get the expected result.

Log file from control:

ERROR:django.request:Internal Server Error: /RPC2
Traceback (most recent call last):
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/srv/lava/.cache/eggs/Django-1.4.2-py2.7.egg/django/views/decorators/csrf.py", line 77, in wrapped_view
    return view_func(*args, **kwargs)
  File "/srv/lava/.cache/eggs/linaro_django_xmlrpc-0.6-py2.7.egg/linaro_django_xmlrpc/views.py", line 87, in handler
    result = dispatcher.marshalled_dispatch(raw_data, user, request)
  File "/srv/lava/.cache/eggs/linaro_django_xmlrpc-0.6-py2.7.egg/linaro_django_xmlrpc/models.py", line 393, in marshalled_dispatch
    response, methodresponse=1, allow_none=self.allow_none)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1085, in dumps
    data = m.dumps(params)
  File "/usr/lib/python2.7/xmlrpclib.py", line 632, in dumps
    dump(v, write)
  File "/usr/lib/python2.7/xmlrpclib.py", line 654, in __dump
    f(self, value, write)
  File "/usr/lib/python2.7/xmlrpclib.py", line 733, in dump_struct
    raise TypeError, "dictionary key must be string"
TypeError: dictionary key must be string

[Matthew Hart (matthew-hart)]

This temporarily went away this morning, so I put my script into salt and the munin graphs were working.

Now they've just stopped and it's the same problem
xmlrpclib.ProtocolError: <ProtocolError for validation.linaro.org/RPC2: 500 INTERNAL SERVER ERROR>

[Neil Williams (codehelp)]

This results from the submitted jobs query returning a None for the job under certain conditions. This results in a null device_type which is not being checked.

A check like:

+ jobs = {}
+ jobs_hash = dict(jobs_res)
+ for job in jobs_hash:
+ if job:
+ jobs[job] = jobs_hash[job]

fixes the problem.