glance registry v2 image-list doesn't work due to bad JSON

[Sam Morrison]

I can't get the glance-registry v2 image-list to work

I've set up rpdb and have found the problem (due to bug 1246968 it's really hard to debug)

When glance-api serialises the body of the request (in common.rpc bulk_request) to send to the registry it has the following data:

[{'command': 'image_get_all', 'kwargs': {'sort_dir': 'desc', 'filters': MultiDict([('deleted', False)]), 'sort_key': 'created_at', 'admin_as_user': False, 'limit': 20, 'member_status': 'accepted', 'marker': None, 'is_public': None}}]

It runs json.dumps(data, default=self._sanitizer) on it and fails with:

*** ValueError: Circular reference detected

The problem is it can't serialise the 'filters': MultiDict([('deleted', False)]), part

Glance-reg v1 works fine but can't get glance-reg v2 working at all

[Sam Morrison (sorrison)]

OK I have a stack trace!

2013-11-11 16:06:54.905 12690 DEBUG eventlet.wsgi.server [4e91620a-a915-4973-bc8a-db6964d5e671 0a01a4d0aa5b430f9dd8a7a0ea80b927 6f1a8103302543d592964de0c826ccae] Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 389, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/glance/glance/common/wsgi.py", line 380, in __call__
    response = req.get_response(self.application)
  File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
    application, catch_exc_info=False)
  File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 571, in __call__
    return self.app(env, start_response)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/glance/glance/common/wsgi.py", line 380, in __call__
    response = req.get_response(self.application)
  File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
    application, catch_exc_info=False)
  File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 1260, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/glance/glance/common/wsgi.py", line 612, in __call__
    request, **action_args)
  File "/opt/glance/glance/common/wsgi.py", line 631, in dispatch
    return method(*args, **kwargs)
  File "/opt/glance/glance/api/v2/images.py", line 84, in index
    member_status=member_status)
  File "/opt/glance/glance/api/authorization.py", line 71, in list
    images = self.image_repo.list(*args, **kwargs)
  File "/opt/glance/glance/domain/proxy.py", line 55, in list
    items = self.base.list(*args, **kwargs)
  File "/opt/glance/glance/api/policy.py", line ...

[Erno Kuvaja (jokke)]

I was able to work around this issue. So indeed that MultiDict is causing some grief here.

Will try to find correct way to patch this.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/79985

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/79985
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=c37c6973859ff45a83de9f35e4b5ebaad894bff4
Submitter: Jenkins
Branch: master

commit c37c6973859ff45a83de9f35e4b5ebaad894bff4
Author: Erno Kuvaja <email address hidden>
Date: Wed Mar 12 16:12:17 2014 +0000

    Detect MultiDict when generating json body

    Detects webob.multidict Multidict at sanitizerand returns mixed()
    out of them so they can be parsed as json.

    Fixes image-list when using Image API v2 with registry.

    Implements a test to verify that the /rpc endpoint exists as
    intended.

    Closes bug: #1246980

    Change-Id: I3d28ca5d8f1af0ec7b9e7a8684e3160f674b8f4c