when using qpid, metadata arguments cause volume cloning to fail

[Giulio Fidente]

volume cloning seems to fail when using qpid if some metadata is passed, see the following:

 # cinder create --metadata 'Type=Test' --source-volid d1ce1fc3-10fe-475f-9dc4-51261d04c323 1
 ERROR: The server has either erred or is incapable of performing the requested operation.

interestingly, this does not seem to happen when a new (non clone) volume is created

cinder api.log reports some rather long traceback:

 InternalError: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 511, in dispatch
    self.engine.dispatch()
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 815, in dispatch
    self.process(ssn)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1050, in process
    self.send(snd, msg)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1261, in send
    body = enc(msg.content)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28, in encode
    sc.write_primitive(type, x)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
    sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
    sc.write_primitive(type, v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
    sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
    sc.write_primitive(type, v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
    sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
    sc.write_primitive(type, v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
    sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
    sc.write_primitive(type, v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 300, in write_list
    type = self.encoding(o)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 59, in encoding
    raise CodecException("no encoding for %r" % obj)
 CodecException: no encoding for <cinder.db.sqlalchemy.models.VolumeMetadata object at 0x458e610>

[Giulio Fidente (gfidente)]

there seems to be a similar report for nova network, see https://bugs.launchpad.net/nova/+bug/1143584

[Eric Harney (eharney)]

The basic problem here is that an object (VolumeMetadata) is being serialized into an AMQP message directly, but it needs to be converted first with something like jsonutils.to_primitive().

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/45035

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/45651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/45651
Committed: http://github.com/openstack/cinder/commit/587eb5a6b9acc9f02eee1147dbaafb3782c295d5
Submitter: Jenkins
Branch: master

commit 587eb5a6b9acc9f02eee1147dbaafb3782c295d5
Author: Flaper Fesp <email address hidden>
Date: Wed Sep 4 13:29:22 2013 +0200

    Call to_primitive on volumes.rpcapi.create_volume

    cinder.volume.rpcapi.create_volume does not convert the request_spec to
    primitive before casting the request. This makes requests containing non
    primitive types to fail. For example:

    cinder create --metadata=Type=test --source-volid $VOLID 1

    This will create a new database record and call create_volume on
    volume.rpcapi. This will fail because VolumeMetadata won't be
    serialized correctly when calling cast. This, however, is not True when
    --source-volid is not passed because in such case,
    scheduler.rpcpai.create_volume will be called, which converts
    request_spec to primitive correctly.

    Closes-Bug: #1213964
    Change-Id: I096d815254c9782390fba05ea9cd9af925635402

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/grizzly
Review: https://review.openstack.org/45861

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/45861
Committed: http://github.com/openstack/cinder/commit/a2a01b1241c8331131a96a870fbbc138fb95a97a
Submitter: Jenkins
Branch: stable/grizzly

commit a2a01b1241c8331131a96a870fbbc138fb95a97a
Author: Flaper Fesp <email address hidden>
Date: Wed Sep 4 13:29:22 2013 +0200

    Call to_primitive on volumes.rpcapi.create_volume

    cinder.volume.rpcapi.create_volume does not convert the request_spec to
    primitive before casting the request. This makes requests containing non
    primitive types to fail. For example:

    cinder create --metadata=Type=test --source-volid $VOLID 1

    This will create a new database record and call create_volume on
    volume.rpcapi. This will fail because VolumeMetadata won't be
    serialized correctly when calling cast. This, however, is not True when
    --source-volid is not passed because in such case,
    scheduler.rpcpai.create_volume will be called, which converts
    request_spec to primitive correctly.

    Closes-Bug 1213964

    Conflicts:
     cinder/volume/rpcapi.py

    (cherry-picked: 587eb5a6b9acc9f02eee1147dbaafb3782c295d5)

    Change-Id: I4b91687cc1e5c933bdb3b04d8560b0500da26541