I got similar but different logs, is your fix able to fix this exception?
2014-01-18 03:41:03.919 15091 INFO nova.compute.manager [-] [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] NV-E7B9660 Destroying instance with name label 'xw00020' which is marked as DELETED but still present on host.
2014-01-18 03:41:03.919 15091 AUDIT nova.compute.manager [req-aef06534-ada2-4751-833f-fc99715e331e None None] [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] NV-067140C Terminating instance
2014-01-18 03:41:04.126 15091 ERROR nova.network.api [-] [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] Failed storing info cache
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] Traceback (most recent call last):
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 83, in update_instance_cache_with_nw_info
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] ic.save(update_cells=update_cells)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 143, in wrapper
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] ctxt, self, fn.__name__, args, kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/conductor/rpcapi.py", line 474, in object_action
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] objmethod=objmethod, args=args, kwargs=kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 85, in call
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return self._invoke(self.proxy.call, ctxt, method, **kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/rpcclient.py", line 63, in _invoke
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return cast_or_call(ctxt, msg, **self.kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/proxy.py", line 126, in call
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] result = rpc.call(context, real_topic, msg, timeout)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py", line 139, in call
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return _get_impl().call(CONF, context, topic, msg, timeout)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 783, in call
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] rpc_amqp.get_connection_pool(conf, Connection))
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 574, in call
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] rv = list(rv)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] raise result
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] InstanceInfoCacheNotFound_Remote: Info cache for instance 6b223492-b954-4d17-9481-bfcc6208e4f6 could not be found.
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] Traceback (most recent call last):
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/conductor/manager.py", line 576, in _object_dispatch
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return getattr(target, method)(context, *args, **kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 152, in wrapper
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return fn(self, ctxt, *args, **kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/objects/instance_info_cache.py", line 91, in save
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] {'network_info': nw_info_json})
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/db/api.py", line 863, in instance_info_cache_update
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return IMPL.instance_info_cache_update(context, instance_uuid, values)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 130, in wrapper
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] return f(*args, **kwargs)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 2343, in instance_info_cache_update
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] instance_uuid=instance_uuid)
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] InstanceInfoCacheNotFound: Info cache for instance 6b223492-b954-4d17-9481-bfcc6208e4f6 could not be found.
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6]
2014-01-18 03:41:04.128 15091 WARNING nova.compute.manager [-] [instance: 6b223492-b954-4d17-9481-bfcc6208e4f6] NV-8907E85 Periodic cleanup failed to delete instance: Info cache for instance 6b223492-b954-4d17-9481-bfcc6208e4f6 could not be found.
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/nova/conductor/manager.py", line 576, in _object_dispatch
return getattr(target, method)(context, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/nova/objects/base.py", line 152, in wrapper
return fn(self, ctxt, *args, **kwargs)
File "/usr/lib/python2.6/site-packages/nova/objects/instance_info_cache.py", line 91, in save
{'network_info': nw_info_json})
File "/usr/lib/python2.6/site-packages/nova/db/api.py", line 863, in instance_info_cache_update
return IMPL.instance_info_cache_update(context, instance_uuid, values)
File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 130, in wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 2343, in instance_info_cache_update
instance_uuid=instance_uuid)
InstanceInfoCacheNotFound: Info cache for instance 6b223492-b954-4d17-9481-bfcc6208e4f6 could not be found.
I got similar but different logs, is your fix able to fix this exception?
2014-01-18 03:41:03.919 15091 INFO nova.compute. manager [-] [instance: 6b223492- b954-4d17- 9481-bfcc6208e4 f6] NV-E7B9660 Destroying instance with name label 'xw00020' which is marked as DELETED but still present on host. manager [req-aef06534- ada2-4751- 833f-fc99715e33 1e None None] [instance: 6b223492- b954-4d17- 9481-bfcc6208e4 f6] NV-067140C Terminating instance b954-4d17- 9481-bfcc6208e4 f6] Failed storing info cache b954-4d17- 9481-bfcc6208e4 f6] Traceback (most recent call last): b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/network/ api.py" , line 83, in update_ instance_ cache_with_ nw_info b954-4d17- 9481-bfcc6208e4 f6] ic.save( update_ cells=update_ cells) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/objects/ base.py" , line 143, in wrapper b954-4d17- 9481-bfcc6208e4 f6] ctxt, self, fn.__name__, args, kwargs) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/conductor/ rpcapi. py", line 474, in object_action b954-4d17- 9481-bfcc6208e4 f6] objmethod= objmethod, args=args, kwargs=kwargs) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/rpcclient. py", line 85, in call b954-4d17- 9481-bfcc6208e4 f6] return self._invoke( self.proxy. call, ctxt, method, **kwargs) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/rpcclient. py", line 63, in _invoke b954-4d17- 9481-bfcc6208e4 f6] return cast_or_call(ctxt, msg, **self.kwargs) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/openstack/ common/ rpc/proxy. py", line 126, in call b954-4d17- 9481-bfcc6208e4 f6] result = rpc.call(context, real_topic, msg, timeout) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/openstack/ common/ rpc/__init_ _.py", line 139, in call b954-4d17- 9481-bfcc6208e4 f6] return _get_impl( ).call( CONF, context, topic, msg, timeout) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/openstack/ common/ rpc/impl_ qpid.py" , line 783, in call b954-4d17- 9481-bfcc6208e4 f6] rpc_amqp. get_connection_ pool(conf, Connection)) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/openstack/ common/ rpc/amqp. py", line 574, in call b954-4d17- 9481-bfcc6208e4 f6] rv = list(rv) b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/openstack/ common/ rpc/amqp. py", line 539, in __iter__ b954-4d17- 9481-bfcc6208e4 f6] raise result b954-4d17- 9481-bfcc6208e4 f6] InstanceInfoCac heNotFound_ Remote: Info cache for instance 6b223492- b954-4d17- 9481-bfcc6208e4 f6 could not be found. b954-4d17- 9481-bfcc6208e4 f6] Traceback (most recent call last): b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/conductor/ manager. py", line 576, in _object_dispatch b954-4d17- 9481-bfcc6208e4 f6] return getattr(target, method)(context, *args, **kwargs) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/objects/ base.py" , line 152, in wrapper b954-4d17- 9481-bfcc6208e4 f6] return fn(self, ctxt, *args, **kwargs) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/objects/ instance_ info_cache. py", line 91, in save b954-4d17- 9481-bfcc6208e4 f6] {'network_info': nw_info_json}) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/db/ api.py" , line 863, in instance_ info_cache_ update b954-4d17- 9481-bfcc6208e4 f6] return IMPL.instance_ info_cache_ update( context, instance_uuid, values) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/db/ sqlalchemy/ api.py" , line 130, in wrapper b954-4d17- 9481-bfcc6208e4 f6] return f(*args, **kwargs) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] File "/usr/lib/ python2. 6/site- packages/ nova/db/ sqlalchemy/ api.py" , line 2343, in instance_ info_cache_ update b954-4d17- 9481-bfcc6208e4 f6] instance_ uuid=instance_ uuid) b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] InstanceInfoCac heNotFound: Info cache for instance 6b223492- b954-4d17- 9481-bfcc6208e4 f6 could not be found. b954-4d17- 9481-bfcc6208e4 f6] b954-4d17- 9481-bfcc6208e4 f6] manager [-] [instance: 6b223492- b954-4d17- 9481-bfcc6208e4 f6] NV-8907E85 Periodic cleanup failed to delete instance: Info cache for instance 6b223492- b954-4d17- 9481-bfcc6208e4 f6 could not be found.
2014-01-18 03:41:03.919 15091 AUDIT nova.compute.
2014-01-18 03:41:04.126 15091 ERROR nova.network.api [-] [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.126 15091 TRACE nova.network.api [instance: 6b223492-
2014-01-18 03:41:04.128 15091 WARNING nova.compute.
Traceback (most recent call last):
File "/usr/lib/ python2. 6/site- packages/ nova/conductor/ manager. py", line 576, in _object_dispatch
return getattr(target, method)(context, *args, **kwargs)
File "/usr/lib/ python2. 6/site- packages/ nova/objects/ base.py" , line 152, in wrapper
return fn(self, ctxt, *args, **kwargs)
File "/usr/lib/ python2. 6/site- packages/ nova/objects/ instance_ info_cache. py", line 91, in save info': nw_info_json})
{'network_
File "/usr/lib/ python2. 6/site- packages/ nova/db/ api.py" , line 863, in instance_ info_cache_ update info_cache_ update( context, instance_uuid, values)
return IMPL.instance_
File "/usr/lib/ python2. 6/site- packages/ nova/db/ sqlalchemy/ api.py" , line 130, in wrapper
return f(*args, **kwargs)
File "/usr/lib/ python2. 6/site- packages/ nova/db/ sqlalchemy/ api.py" , line 2343, in instance_ info_cache_ update uuid=instance_ uuid)
instance_
InstanceInfoCac heNotFound: Info cache for instance 6b223492- b954-4d17- 9481-bfcc6208e4 f6 could not be found.
Best Regards!