Cannot create image: Json Circular reference detected

[Chmouel Boudjnah]

When trying to play around with 'glance' and create an image I am keep getting a :

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/local/lib/python2.6/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/pymodules/python2.6/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/local/lib/python2.6/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/local/lib/python2.6/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/local/lib/python2.6/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/home/chmouel/GIT/glance/glance/common/wsgi.py", line 217, in __call__
    return self._serialize(result, req)
  File "/home/chmouel/GIT/glance/glance/common/wsgi.py", line 229, in _serialize
    return serializer.to_content_type(data)
  File "/home/chmouel/GIT/glance/glance/common/wsgi.py", line 259, in to_content_type
    return self._methods.get(mimetype, repr)(data)
  File "/home/chmouel/GIT/glance/glance/common/wsgi.py", line 268, in _to_json
    return json.dumps(data, default=sanitizer)
  File "/usr/lib/python2.6/json/__init__.py", line 237, in dumps
    **kw).encode(obj)
  File "/usr/lib/python2.6/json/encoder.py", line 367, in encode
    chunks = list(self.iterencode(o))
  File "/usr/lib/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib/python2.6/json/encoder.py", line 275, in _iterencode_dict
    for chunk in self._iterencode(value, markers):
  File "/usr/lib/python2.6/json/encoder.py", line 317, in _iterencode
    for chunk in self._iterencode_default(o, markers):
  File "/usr/lib/python2.6/json/encoder.py", line 315, in _iterencode
    raise ValueError("Circular reference detected")
ValueError: Circular reference detected

Something along those lines would fix it : (since we were trying to serialize a non serializable object)

=== modified file 'glance/parallax/controllers.py'
--- glance/parallax/controllers.py 2010-12-02 20:19:26 +0000
+++ glance/parallax/controllers.py 2010-12-07 10:57:59 +0000
@@ -108,6 +108,9 @@
         try:
             new_image = db.image_create(context, image_data)
             return dict(image=new_image)
+ n = {'image' : dict(new_image)}
+ del(n['image']['_sa_instance_state'])
+ return n
         except exception.Duplicate:
             return exc.HTTPConflict()
         except exception.Invalid:

Let me know if that seems to be good enough and I'll do a merge request..

[Chmouel Boudjnah (chmouel)]

More background on that, this is my test script:

,----
| import httplib
| import json
| cnx = httplib.HTTPConnection("chmoutest", 9191)
| image_metadata = {'id': 2,
| 'name': 'fake image #2',
| 'is_public': True,
| 'image_type': 'kernel',
| 'status': 'available',
| 'properties': []}
| if 'image' not in image_metadata.keys():
| image_metadata = dict(image=image_metadata)
|
| body = json.dumps(image_metadata)
| print json.loads(body)
|
| cnx.request("POST", "/images", body)
|
| res = cnx.getresponse()
| print res.status
| print res.read()
`----

this is the object trying to get serialized without my patch :

,----
| {'image': <glance.parallax.db.sqlalchemy.models.Image object at 0x1d50d10>}
`----

when this should be probably this object ( the n = {'image' : dict(new_image)} part ):

,----
| {'image': {'_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x1f54e10>,
| 'created_at': datetime.datetime(2010, 12, 9, 10, 22, 48, 311160),
| 'deleted': False,
| 'deleted_at': None,
| 'id': 2,
| 'image_type': u'kernel',
| 'is_public': True,
| 'name': u'fake image #2',
| 'properties': [],
| 'status': u'available',
| 'updated_at': None}}
`----

maybe I am not using it properly ?

I have ended doing the insert directly in the DB without the API :

http://c0403512.cdn.cloudfiles.rackspacecloud.com/glance_create_data_noauth.py

[Jay Pipes (jaypipes)]

Ah, I see now what you are doing. :)

No, that is not the correct way to work with Glance :) You should never need to work with the SQLAlchemy or even the db API. Here is how you can add an image's metadata to parallax:

from glance import client
from glance.common import exception

c = client.ParallaxClient("http://127.0.0.1", 9191)

image_metadata = {'name': 'fake image #2',
| 'is_public': True,
| 'image_type': 'kernel',
| 'properties': []}

# Note above:no need to specify the status or the id. Glance generates those for you.

try:
    result = c.add_image(image_metadata)
except exception.Duplicate:
   # blah
except exception.BadRequest:
   # blah...

Hope this helps! I'm working on documenting all this stuff. Be aware that there are some significant changes coming, too. I'll keep you informed.
jay

[Jay Pipes (jaypipes)]

Chmouel, please set to Not a Bug if the above worked for you :)

[Chmouel Boudjnah (chmouel)]

oh ok, I basically copied the code from client on my first script when accessing directly via HTTP. I am not near my test machine at the moment, i'll test it if it works and would set as not a bug if it does.

Chmouel.

[Rick Harris (rconradharris)]

I was able to replicate this. I'm thinking the solution here is to have db/sqlalchemy/api sanitize out any sqlalchemy specific information (in this case _sa_instance_state) before returning the object.

If we don't , that means our db abstraction would be leaking SQLAlchemy specific implementation details. This would have the side-effect of fixing this issue as well.

[Jay Pipes (jaypipes)]

I'm working on code that will do away with this problem... gimme a few hours.

[Thierry Carrez (ttx)]

Jay, what's the status of that bug ?

[Jay Pipes (jaypipes)]

Should be resolved now. Fix Committed.

On Fri, Jan 28, 2011 at 3:48 AM, Thierry Carrez <email address hidden> wrote:
> Jay, what's the status of that bug ?
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/686438
>
> Title:
>  Cannot create image: Json Circular reference detected
>