OpenStack Compute (nova)

Creating network failed with message handling via REST API

Bug #1143584 reported by Fan Zhang on 2013-03-04

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	High	Hua Zhang	OpenStack Compute (nova) 2013.1 "grizzly"
	Folsom	Fix Released	High	Nikola Đipanov	OpenStack Compute (nova) 2012.2.4

Bug Description

Test steps:
1. get tokens successfully:
token:<1f0d9345795f413380c662934277cf67>
tenant:<0dfe9d0e4a9347a0a9553800ac8a05d9>

2. do request for creating a network:
POST:http://9.181.137.197:8774/v2/0dfe9d0e4a9347a0a9553800ac8a05d9/os-networks
body:
{
    "network":
       {
            "cidr": "9.71.105.0/24",
            "label": "mynetwork",
            "multi_host": "false"
        }
}
Results:
Return 500 error like "{"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}"
1. check network list, find the network actually created successfully to add to list.
2. why the task status is error? The error message from network.log as below:
2013-01-28 13:54:10.755 ERROR nova.openstack.common.rpc.amqp [req-43f72cf1-df74-4580-81da-90377b7b2d0c 5b8a7abfedb944b1987227ae12c0e16e 0dfe9d0e4a9347a0a9553800ac8a05d9] Exception during message handling
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 282, in _process_data
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp ctxt.reply(rval, None, connection_pool=self.connection_pool)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 191, in reply
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp ending, log_failure)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 171, in msg_reply
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp conn.direct_send(msg_id, rpc_common.serialize_msg(msg))
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 124, in __exit__
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self._done()
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 113, in _done
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.connection.reset()
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 387, in reset
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.session.close()
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "<string>", line 6, in close
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 739, in close
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.sync(timeout=timeout)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "<string>", line 6, in sync
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 730, in sync
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp snd.sync(timeout=timeout)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "<string>", line 6, in sync
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 885, in sync
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp if not self._ewait(lambda: self.acked >= mno, timeout=timeout):
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 799, in _ewait
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp result = self.session._ewait(lambda: self.error or predicate(), timeout)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 566, in _ewait
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp result = self.connection._ewait(lambda: self.error or predicate(), timeout)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 209, in _ewait
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.check_error()
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 202, in check_error
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp raise self.error
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp InternalError: Traceback (most recent call last):
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 511, in dispatch
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.engine.dispatch()
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 817, in dispatch
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.process(ssn)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1052, in process
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp self.send(snd, msg)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1263, in send
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp body = enc(msg.content)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28, in encode
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp sc.write_primitive(type, x)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp getattr(self, "write_%s" % type.NAME)(v)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp sc.write_primitive(type, v)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp getattr(self, "write_%s" % type.NAME)(v)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 300, in write_list
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp type = self.encoding(o)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 59, in encoding
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp raise CodecException("no encoding for %r" % obj)
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp CodecException: no encoding for <nova.db.sqlalchemy.models.Network object at 0x4a8ea10>
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp
2013-01-28 13:54:10.755 12667 TRACE nova.openstack.common.rpc.amqp
2013-01-28 13:54:11.402 INFO nova.openstack.common.rpc.impl_qpid [req-43f72cf1-df74-4580-81da-90377b7b2d0c 5b8a7abfedb944b1987227ae12c0e16e 0dfe9d0e4a9347a0a9553800ac8a05d9] Connected to AMQP server on localhost:5672
2013-01-28 13:54:11.404 ERROR nova.openstack.common.rpc.common [req-43f72cf1-df74-4580-81da-90377b7b2d0c 5b8a7abfedb944b1987227ae12c0e16e 0dfe9d0e4a9347a0a9553800ac8a05d9] Returning exception Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 511, in dispatch
    self.engine.dispatch()
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 817, in dispatch
    self.process(ssn)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1052, in process
    self.send(snd, msg)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1263, in send
    body = enc(msg.content)
  File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28, in encode
    sc.write_primitive(type, x)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map
    sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem
    sc.write_primitive(type, v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive
    getattr(self, "write_%s" % type.NAME)(v)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 300, in write_list
    type = self.encoding(o)
  File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 59, in encoding
    raise CodecException("no encoding for %r" % obj)
CodecException: no encoding for <nova.db.sqlalchemy.models.Network object at 0x4a8ea10>
to caller
2013-01-28 13:54:11.404 ERROR nova.openstack.common.rpc.common [req-43f72cf1-df74-4580-81da-90377b7b2d0c 5b8a7abfedb944b1987227ae12c0e16e 0dfe9d0e4a9347a0a9553800ac8a05d9] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 282, in _process_data\n ctxt.reply(rval, None, connection_pool=self.connection_pool)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 191, in reply\n ending, log_failure)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 171, in msg_reply\n conn.direct_send(msg_id, rpc_common.serialize_msg(msg))\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 124, in __exit__\n self._done()\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 113, in _done\n self.connection.reset()\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 387, in reset\n self.session.close()\n', ' File "<string>", line 6, in close\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 739, in close\n self.sync(timeout=timeout)\n', ' File "<string>", line 6, in sync\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 730, in sync\n snd.sync(timeout=timeout)\n', ' File "<string>", line 6, in sync\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 885, in sync\n if not self._ewait(lambda: self.acked >= mno, timeout=timeout):\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 799, in _ewait\n result = self.session._ewait(lambda: self.error or predicate(), timeout)\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 566, in _ewait\n result = self.connection._ewait(lambda: self.error or predicate(), timeout)\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 209, in _ewait\n self.check_error()\n', ' File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line 202, in check_error\n raise self.error\n', 'InternalError: Traceback (most recent call last):\n File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 511, in dispatch\n self.engine.dispatch()\n File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 817, in dispatch\n self.process(ssn)\n File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1052, in process\n self.send(snd, msg)\n File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line 1263, in send\n body = enc(msg.content)\n File "/usr/lib/python2.6/site-packages/qpid/messaging/message.py", line 28, in encode\n sc.write_primitive(type, x)\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive\n getattr(self, "write_%s" % type.NAME)(v)\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 257, in write_map\n sc.write(string.joinfields(map(self._write_map_elem, m.keys(), m.values()), ""))\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 250, in _write_map_elem\n sc.write_primitive(type, v)\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 73, in write_primitive\n getattr(self, "write_%s" % type.NAME)(v)\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 300, in write_list\n type = self.encoding(o)\n File "/usr/lib/python2.6/site-packages/qpid/codec010.py", line 59, in encoding\n raise CodecException("no encoding for %r" % obj)\nCodecException: no encoding for <nova.db.sqlalchemy.models.Network object at 0x4a8ea10>\n\n']

Tags:

Revision history for this message

Fan Zhang (bjzhfan) wrote on 2013-03-04:

This problem could be fixed by change "return networks" to "return jsonutils.to_primitive(networks)" in nova/network/manager.py def _do_create_networks() method.

Vish Ishaya (vishvananda) on 2013-03-04

Changed in nova:
status:	New → Triaged
importance:	Undecided → High
milestone:	none → grizzly-rc1
tags:	added: low-hanging-fruit

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-03-05: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/23529

Changed in nova:
assignee:	nobody → Hua Zhang (zhhuabj)
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-03-09: Fix merged to nova (master)

Reviewed: https://review.openstack.org/23529
Committed: http://github.com/openstack/nova/commit/b4d43b0d0c890ccb940862719c6b88a24029d44a
Submitter: Jenkins
Branch: master

commit b4d43b0d0c890ccb940862719c6b88a24029d44a
Author: zhhuabj <email address hidden>
Date: Tue Mar 5 17:39:50 2013 +0800

Fix Network object encoding issue when using qpid

    When creating network via REST API will throw the exception
    "CodecException: no encoding for
    <nova.db.sqlalchemy.models.Network object at 0x4a8ea10>".
    It only occurs when using qpid.
    Fix this issue through using jsonutils.to_primitive method.

Fix bug 1143584.

Change-Id: I833aef451f8e3d5eaa246c19d67b2474a9d36910

Changed in nova:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2013-03-20

Changed in nova:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2013-04-04

Changed in nova:
milestone:	grizzly-rc1 → 2013.1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-04-05: Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/26225

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2013-04-05: Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/26225
Committed: http://github.com/openstack/nova/commit/975a312da9cb8f601e4ffb71ddfa934f5b06abe7
Submitter: Jenkins
Branch: stable/folsom

commit 975a312da9cb8f601e4ffb71ddfa934f5b06abe7
Author: zhhuabj <email address hidden>
Date: Tue Mar 5 17:39:50 2013 +0800

Fix Network object encoding issue when using qpid

Fix bug 1143584.

Change-Id: I833aef451f8e3d5eaa246c19d67b2474a9d36910
(cherry picked from commit b4d43b0d0c890ccb940862719c6b88a24029d44a)

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.