2014-04-04 15:38:16.302 ERROR cinder.volume.flows.manager.create_volume [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] Volume fbf73ff7-5c78-4d41-a30c-b92f14690215: create failed
2014-04-04 15:38:16.303 DEBUG taskflow.storage [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' from (pid=17847) _with_connection /opt/stack/taskflow/taskflow/storage.py:86
2014-04-04 15:38:16.303 DEBUG taskflow.storage [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' from (pid=17847) _with_connection /opt/stack/taskflow/taskflow/storage.py:86
2014-04-04 15:38:16.304 DEBUG taskflow.storage [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_flow_detail' from (pid=17847) _with_connection /opt/stack/taskflow/taskflow/storage.py:86
2014-04-04 15:38:16.304 ERROR oslo.messaging._executors.base [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] Exception during message handling
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base Traceback (most recent call last):
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_executors/base.py", line 36, in _dispatch
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     incoming.reply(self.callback(incoming.ctxt, incoming.message))
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in __call__
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return self._dispatch(endpoint, method, ctxt, args)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 104, in _dispatch
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     result = getattr(endpoint, method)(ctxt, **new_args)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/manager.py", line 363, in create_volume
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     _run_flow()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/manager.py", line 356, in _run_flow
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     flow_engine.run()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/utils/lock_utils.py", line 54, in wrapper
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return f(*args, **kwargs)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/engines/action_engine/engine.py", line 94, in run
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     self._run()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/engines/action_engine/engine.py", line 109, in _run
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     misc.Failure.reraise_if_any(failures.values())
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/utils/misc.py", line 586, in reraise_if_any
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     failures[0].reraise()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/utils/misc.py", line 593, in reraise
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     six.reraise(*self._exc_info)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/taskflow/taskflow/engines/action_engine/executor.py", line 34, in _execute_task
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     result = task.execute(**arguments)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 594, in execute
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     **volume_spec)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 564, in _create_raw_volume
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return self.driver.create_volume(volume_ref)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 262, in create_volume
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     self._verify_volume_creation(volume)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 246, in _verify_volume_creation
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     self._select_ds_for_volume(volume)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 468, in _select_ds_for_volume
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     retrv_result = self.volumeops.get_hosts()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/volumeops.py", line 121, in get_hosts
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     'HostSystem', self._max_objects)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 252, in invoke_api
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return _invoke_api(module, method, *args, **kwargs)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 89, in func
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return timer.wait()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return hubs.get_hub().switch()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     return self.greenlet.switch()
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/openstack/common/loopingcall.py", line 123, in _inner
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     idle = self.f(*self.args, **self.kw)
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base   File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 82, in _func
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base     raise excep
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base VimFaultException: Server raised fault: 'The session is not authenticated.'
2014-04-04 15:38:16.304 TRACE oslo.messaging._executors.base
2014-04-04 15:38:16.306 ERROR oslo.messaging._drivers.common [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] Returning exception Server raised fault: 'The session is not authenticated.' to caller
2014-04-04 15:38:16.307 ERROR oslo.messaging._drivers.common [req-16c7d518-8664-44c3-be49-1b91072fa829 b2fe897439b148ccadde46f9a18494a5 482eacf2be1d4bc199822524ff39a2c4] ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_executors/base.py", line 36, in _dispatch\n    incoming.reply(self.callback(incoming.ctxt, incoming.message))\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in __call__\n    return self._dispatch(endpoint, method, ctxt, args)\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 104, in _dispatch\n    result = getattr(endpoint, method)(ctxt, **new_args)\n', '  File "/opt/stack/cinder/cinder/volume/manager.py", line 363, in create_volume\n    _run_flow()\n', '  File "/opt/stack/cinder/cinder/volume/manager.py", line 356, in _run_flow\n    flow_engine.run()\n', '  File "/opt/stack/taskflow/taskflow/utils/lock_utils.py", line 54, in wrapper\n    return f(*args, **kwargs)\n', '  File "/opt/stack/taskflow/taskflow/engines/action_engine/engine.py", line 94, in run\n    self._run()\n', '  File "/opt/stack/taskflow/taskflow/engines/action_engine/engine.py", line 109, in _run\n    misc.Failure.reraise_if_any(failures.values())\n', '  File "/opt/stack/taskflow/taskflow/utils/misc.py", line 586, in reraise_if_any\n    failures[0].reraise()\n', '  File "/opt/stack/taskflow/taskflow/utils/misc.py", line 593, in reraise\n    six.reraise(*self._exc_info)\n', '  File "/opt/stack/taskflow/taskflow/engines/action_engine/executor.py", line 34, in _execute_task\n    result = task.execute(**arguments)\n', '  File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 594, in execute\n    **volume_spec)\n', '  File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 564, in _create_raw_volume\n    return self.driver.create_volume(volume_ref)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 262, in create_volume\n    self._verify_volume_creation(volume)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 246, in _verify_volume_creation\n    self._select_ds_for_volume(volume)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/vmdk.py", line 468, in _select_ds_for_volume\n    retrv_result = self.volumeops.get_hosts()\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/volumeops.py", line 121, in get_hosts\n    \'HostSystem\', self._max_objects)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 252, in invoke_api\n    return _invoke_api(module, method, *args, **kwargs)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 89, in func\n    return timer.wait()\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait\n    return hubs.get_hub().switch()\n', '  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch\n    return self.greenlet.switch()\n', '  File "/opt/stack/cinder/cinder/openstack/common/loopingcall.py", line 123, in _inner\n    idle = self.f(*self.args, **self.kw)\n', '  File "/opt/stack/cinder/cinder/volume/drivers/vmware/api.py", line 82, in _func\n    raise excep\n', "VimFaultException: Server raised fault: 'The session is not authenticated.'\n"]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenpool.py", line 80, in _spawn_n_impl
    func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_executors/base.py", line 46, in _dispatch
    incoming.reply(failure=exc_info)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 63, in reply
    self._send_reply(conn, reply, failure, log_failure=log_failure)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 44, in _send_reply
    log_failure)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/common.py", line 327, in serialize_remote_exception
    json_data = jsonutils.dumps(data)
  File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/openstack/common/jsonutils.py", line 164, in dumps
    return json.dumps(value, default=default, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
ValueError: Circular reference detected