test_volume_boot_pattern: Server %(server_id)s failed to build and is in ERROR status

[Jamie Lennox]

From: http://logs.openstack.org/66/46866/3/check/gate-tempest-devstack-vm-postgres-full/bae3106/console.html

2013-09-24 01:46:29.250 | ======================================================================
2013-09-24 01:46:29.250 | FAIL: tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern[compute,image,volume]
2013-09-24 01:46:29.251 | tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern[compute,image,volume]
2013-09-24 01:46:29.251 | ----------------------------------------------------------------------
2013-09-24 01:46:29.251 | _StringException: Empty attachments:
2013-09-24 01:46:29.251 | stderr
2013-09-24 01:46:29.251 | stdout
2013-09-24 01:46:29.251 |
2013-09-24 01:46:29.251 | pythonlogging:'': {{{
2013-09-24 01:46:29.252 | 2013-09-24 01:40:42,284 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:42,406 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:43,447 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:43,868 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:46,900 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:47,980 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.252 | 2013-09-24 01:40:49,059 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.253 | 2013-09-24 01:40:50,138 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.253 | 2013-09-24 01:41:49,730 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.253 | 2013-09-24 01:41:50,904 Authentication (publickey) failed.
2013-09-24 01:46:29.253 | 2013-09-24 01:41:52,546 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.253 | 2013-09-24 01:41:53,217 Authentication (publickey) successful!
2013-09-24 01:46:29.253 | 2013-09-24 01:41:53,399 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.254 | 2013-09-24 01:41:54,059 Authentication (publickey) successful!
2013-09-24 01:46:29.254 | 2013-09-24 01:41:54,066 Secsh channel 1 opened.
2013-09-24 01:46:29.254 | 2013-09-24 01:41:54,159 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.254 | 2013-09-24 01:41:54,819 Authentication (publickey) successful!
2013-09-24 01:46:29.254 | 2013-09-24 01:41:54,825 Secsh channel 1 opened.
2013-09-24 01:46:29.254 | 2013-09-24 01:42:53,239 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.254 | 2013-09-24 01:42:54,079 Authentication (publickey) successful!
2013-09-24 01:46:29.255 | 2013-09-24 01:42:54,195 Connected (version 2.0, client dropbear_2012.55)
2013-09-24 01:46:29.255 | 2013-09-24 01:42:54,924 Authentication (publickey) successful!
2013-09-24 01:46:29.255 | 2013-09-24 01:42:54,930 Secsh channel 1 opened.
2013-09-24 01:46:29.255 | 2013-09-24 01:42:54,980 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.255 | 2013-09-24 01:42:55,070 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.255 | 2013-09-24 01:42:56,110 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.255 | 2013-09-24 01:42:56,159 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | 2013-09-24 01:42:56,319 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | 2013-09-24 01:42:57,399 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | 2013-09-24 01:42:58,544 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | 2013-09-24 01:42:59,669 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | 2013-09-24 01:43:00,776 Starting new HTTP connection (1): 127.0.0.1
2013-09-24 01:46:29.256 | }}}
2013-09-24 01:46:29.257 |
2013-09-24 01:46:29.257 | Traceback (most recent call last):
2013-09-24 01:46:29.257 | File "tempest/scenario/test_volume_boot_pattern.py", line 154, in test_volume_boot_pattern
2013-09-24 01:46:29.257 | keypair)
2013-09-24 01:46:29.257 | File "tempest/scenario/test_volume_boot_pattern.py", line 53, in _boot_instance_from_volume
2013-09-24 01:46:29.257 | create_kwargs=create_kwargs)
2013-09-24 01:46:29.257 | File "tempest/scenario/manager.py", line 390, in create_server
2013-09-24 01:46:29.258 | self.status_timeout(client.servers, server.id, 'ACTIVE')
2013-09-24 01:46:29.258 | File "tempest/scenario/manager.py", line 290, in status_timeout
2013-09-24 01:46:29.258 | self._status_timeout(things, thing_id, expected_status=expected_status)
2013-09-24 01:46:29.258 | File "tempest/scenario/manager.py", line 338, in _status_timeout
2013-09-24 01:46:29.258 | self.config.compute.build_interval):
2013-09-24 01:46:29.258 | File "tempest/test.py", line 237, in call_until_true
2013-09-24 01:46:29.259 | if func():
2013-09-24 01:46:29.259 | File "tempest/scenario/manager.py", line 329, in check_status
2013-09-24 01:46:29.259 | raise exceptions.BuildErrorException(message)
2013-09-24 01:46:29.259 | BuildErrorException: Server %(server_id)s failed to build and is in ERROR status
2013-09-24 01:46:29.259 | Details: <Server: scenario-server-725246790> failed to get to expected status. In ERROR state.
2013-09-24 01:46:29.259 |
2013-09-24 01:46:29.259 |
2013-09-24 01:46:29.260 | ======================================================================
2013-09-24 01:46:29.260 | FAIL: process-returncode
2013-09-24 01:46:29.260 | process-returncode
2013-09-24 01:46:29.260 | ----------------------------------------------------------------------
2013-09-24 01:46:29.260 | _StringException: Binary content:
2013-09-24 01:46:29.260 | traceback (test/plain; charset="utf8")

Possibly coming from: http://logs.openstack.org/66/46866/3/check/gate-tempest-devstack-vm-postgres-full/bae3106/logs/screen-n-cpu.txt.gz

d005876-0642-41c9-9a20-8f7834c4c49e ServersNegativeTestJSON899026748-user ServersNegativeTestJSON205799752-tenant] Possibly task preempted.
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager Traceback (most recent call last):
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return function(self, context, *args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 295, in decorated_function
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager function(self, context, *args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager e, sys.exc_info())
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return function(self, context, *args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 1614, in run_instance
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager do_run_instance()
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/lockutils.py", line 246, in inner
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return f(*args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 1613, in do_run_instance
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager legacy_bdm_in_spec)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 963, in _run_instance
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager notify("error", msg=unicode(e)) # notify that build failed
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 938, in _run_instance
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager image_meta = self._prebuild_instance(context, instance)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 969, in _prebuild_instance
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager self._start_building(context, instance)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 1204, in _start_building
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager None))
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/compute/manager.py", line 474, in _instance_update
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/conductor/api.py", line 412, in instance_update
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager updates, 'conductor')
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/conductor/rpcapi.py", line 138, in instance_update
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager service=service)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/rpcclient.py", line 85, in call
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return self._invoke(self.proxy.call, ctxt, method, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/rpcclient.py", line 63, in _invoke
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return cast_or_call(ctxt, msg, **self.kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/proxy.py", line 126, in call
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager result = rpc.call(context, real_topic, msg, timeout)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/__init__.py", line 139, in call
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return _get_impl().call(CONF, context, topic, msg, timeout)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager rpc_amqp.get_connection_pool(conf, Connection))
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 574, in call
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager rv = list(rv)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager raise result
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager UnexpectedTaskStateError_Remote: unexpected task state: expecting [u'scheduling', None] but the actual state is deleting
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager Traceback (most recent call last):
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/openstack/common/rpc/common.py", line 420, in catch_client_exception
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return func(*args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/conductor/manager.py", line 133, in instance_update
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager context, instance_uuid, updates)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/db/api.py", line 704, in instance_update_and_get_original
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager columns_to_join=columns_to_join)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 126, in wrapper
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager return f(*args, **kwargs)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 2168, in instance_update_and_get_original
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager columns_to_join=columns_to_join)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 2214, in _instance_update
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager expected=expected)
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager UnexpectedTaskStateError: unexpected task state: expecting [u'scheduling', None] but the actual state is deleting
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.333 22622 TRACE nova.compute.manager
2013-09-24 01:34:27.339 ERROR nova.openstack.common.rpc.amqp [req-7d005876-0642-41c9-9a20-8f7834c4c49e ServersNegativeTestJSON899026748-user ServersNegativeTestJSON205799752-tenant] Exception during message handling
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp **args)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/exception.py", line 90, in wrapped
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp payload)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/exception.py", line 73, in wrapped
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return f(self, context, *args, **kw)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 236, in decorated_function
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp LOG.exception(_("Possibly task preempted."))
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 230, in decorated_function
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 295, in decorated_function
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp function(self, context, *args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 272, in decorated_function
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp e, sys.exc_info())
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 259, in decorated_function
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 1614, in run_instance
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp do_run_instance()
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/lockutils.py", line 246, in inner
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 1613, in do_run_instance
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp legacy_bdm_in_spec)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 963, in _run_instance
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp notify("error", msg=unicode(e)) # notify that build failed
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 938, in _run_instance
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp image_meta = self._prebuild_instance(context, instance)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 969, in _prebuild_instance
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp self._start_building(context, instance)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 1204, in _start_building
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp None))
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/compute/manager.py", line 474, in _instance_update
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/conductor/api.py", line 412, in instance_update
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp updates, 'conductor')
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/conductor/rpcapi.py", line 138, in instance_update
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp service=service)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/rpcclient.py", line 85, in call
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return self._invoke(self.proxy.call, ctxt, method, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/rpcclient.py", line 63, in _invoke
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return cast_or_call(ctxt, msg, **self.kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/proxy.py", line 126, in call
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp result = rpc.call(context, real_topic, msg, timeout)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/__init__.py", line 139, in call
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return _get_impl().call(CONF, context, topic, msg, timeout)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp rpc_amqp.get_connection_pool(conf, Connection))
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 574, in call
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp rv = list(rv)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp raise result
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError_Remote: unexpected task state: expecting [u'scheduling', None] but the actual state is deleting
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/openstack/common/rpc/common.py", line 420, in catch_client_exception
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return func(*args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/conductor/manager.py", line 133, in instance_update
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp context, instance_uuid, updates)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/db/api.py", line 704, in instance_update_and_get_original
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp columns_to_join=columns_to_join)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 126, in wrapper
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 2168, in instance_update_and_get_original
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp columns_to_join=columns_to_join)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 2214, in _instance_update
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp expected=expected)
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError: unexpected task state: expecting [u'scheduling', None] but the actual state is deleting
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp
2013-09-24 01:34:27.339 22622 TRACE nova.openstack.common.rpc.amqp