Devstack installation with cells, throws 'circular-reference detected' on nova cell-show command

[Vineet Menon]

Devstack installtion with local.conf,

[[local|localrc]]
        ADMIN_PASSWORD=secrete
        DATABASE_PASSWORD=secrete
        RABBIT_PASSWORD=secrete
        SERVICE_PASSWORD=secrete
        SERVICE_TOKEN=a682f596-76f3-11e3-b3b2-e716f9080d50
        #FIXED_RANGE=172.31.1.0/24
        #FLOATING_RANGE=192.168.20.0/25
        #HOST_IP=10.3.4.5

        ENABLED_SERVICES+=,n-cell

installs a devstack with a child cell (with name child) and a parent cell. It's possible to create a server in it.

When issued the command nova cell-show child, it throws exception,

ERROR (ClientException): The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-5df5470a-5135-4cca-b301-238b0a9ce7c2)

The relevant stack trace is as follows,

2014-10-21 13:21:34.375 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Circular reference detected
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 72, in reply
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher self._send_reply(conn, reply, failure, log_failure=log_failure)
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 62, in _send_reply
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", line 302, in serialize_msg
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher _MESSAGE_KEY: jsonutils.dumps(raw_msg)}
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py", line 172, in dumps
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher return json.dumps(value, default=default, **kwargs)
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher sort_keys=sort_keys, **kw).encode(obj)
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher chunks = self.iterencode(o, _one_shot=True)
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher return _iterencode(o, 0)
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher ValueError: Circular reference detected
2014-10-21 13:21:34.375 TRACE oslo.messaging.rpc.dispatcher
2014-10-21 13:21:34.377 ERROR oslo.messaging._drivers.common [-] Returning exception Circular reference detected to caller
2014-10-21 13:21:34.377 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 72, in reply\n self._send_reply(conn, reply, failure, log_failure=log_failure)\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 62, in _send_reply\n conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", line 302, in serialize_msg\n _MESSAGE_KEY: jsonutils.dumps(raw_msg)}\n', ' File "/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py", line 172, in dumps\n return json.dumps(value, default=default, **kwargs)\n', ' File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps\n sort_keys=sort_keys, **kw).encode(obj)\n', ' File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode\n chunks = self.iterencode(o, _one_shot=True)\n', ' File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode\n return _iterencode(o, 0)\n', 'ValueError: Circular reference detected\n']

[Joe Gordon (jogo)]

This sounds like a configuration issue, if you are looking to get a question answered, please use ask.openstack.org

[Vineet Menon (mvineetmenon)]

Well, I don't think this is a configuration problem because instances are getting created with this setup, i.e. I'm able to boot VMs in child.

Anyway, I ahve posted a question in ask.openstack.org. Meanwhile, you may put this bug on hold.

Here are some relevant bugs which have already been submitted,
1. https://bugs.launchpad.net/nova/+bug/1338403
2. https://bugs.launchpad.net/cinder/+bug/1294949
3. http://lists.openstack.org/pipermail/openstack-dev/2014-February/027933.html