v2 API: image create and update return HTTP 500

[Alex Meade]

Any request that performs a notification in v2 of the API fails with an HTTP 500. This is because the image object received from the db has the _sa_instance_state attribute which is not JSON serializable. So when the notifier is called it fails with the following traceback. In most cases however, the actual change in data is performed since notification calls generally happen last.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 382, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 332, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
    app_iter = application(self.environ, start_response)
  File "/opt/stack/keystone/keystone/middleware/auth_token.py", line 278, in __call__
    return self.app(env, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 332, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/pymodules/python2.7/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/opt/stack/glance/glance/common/wsgi.py", line 538, in __call__
    request, **action_args)
  File "/opt/stack/glance/glance/common/wsgi.py", line 555, in dispatch
    return method(*args, **kwargs)
  File "/opt/stack/glance/glance/common/utils.py", line 408, in wrapped
    return func(self, req, *args, **kwargs)
  File "/opt/stack/glance/glance/api/v2/images.py", line 109, in create
    self.notifier.info('image.update', image)
  File "/opt/stack/glance/glance/notifier/__init__.py", line 83, in info
    self.strategy.info(msg)
  File "/opt/stack/glance/glance/notifier/notify_kombu.py", line 221, in info
    self._notify(msg, "INFO")
  File "/opt/stack/glance/glance/notifier/notify_kombu.py", line 195, in _notify
    self._send_message(msg, routing_key)
  File "/opt/stack/glance/glance/notifier/notify_kombu.py", line 176, in _send_message
    msg = self.exchange.Message(json.dumps(msg),
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <sqlalchemy.orm.state.InstanceState object at 0x2d56110> is not JSON serializable

[Iccha Sethi (iccha-sethi)]

This may be addressed by the domain model code.

[Brian Waldon (bcwaldon)]

Alex - can you verify that this is or is not an issue?

[Jeff Marcom (jeffmarcom)]

This seems to very much still be an issue. I'm getting the same error when trying to upload an image using PUT v2/images/<image-id/file:

TypeError: <glance.db.sqlalchemy.models.Image object at 0x3cb61d0> is not JSON serializable

Why was this marked invalid?

[Jeff Marcom (jeffmarcom)]

Btw, I can see that the image I was attempting an upload for was partially saved, however the checksum and file size are incomplete and the transfer is interrupted by the above error:

Name
certification
ID
9e977bf9-d6c1-47c8-9f70-1fe3fd9a66d8
Status
Queued
Public
False
Checksum
a09f6b1ee86059bcd48534bce96282e5
Created
2013-05-05T18:18:54
Updated
Never updated
Specs
Size
553.9 MB
Container Format
OVF
Disk Format
QCOW2

[Alex Meade (alex-meade)]

Setting this to Fix Committed since it not longer seems to be an issue. Looking into it, the domain model code fixed the root of this problem.