terminated instance that won't disappear

[Chris Jones]

I have a nova deployment which is generally functioning normally, however there is an instance which shows as terminated even after several days. The instance itself is gone from the machine it was running on, and "nova list" on that machine doesn't seem to show any relevant instances.

[Brian Waldon (bcwaldon)]

Can you explain what you mean by 'shows as terminated'? What method are you using to reproduce this behavior?

[Chris Jones (cmsj)]

euca-describe-instances shows it as terminated and has done for about 6 days

[Gavin B (gavin-brebner-orange)]

I find that instances stuck as "terminated" typically come from a communication break-down to the host server e.g. if the nova-compute service is not running the terminated instance will stay as terminated until the node come back online.

Could that be the case here ?

[Andrew Glen-Young (aglenyoung)]

@Thierry

The nova-compute service is currently running and the instance is marked as 'terminated'. There are other instances on this compute node which I can create, destroy, etc. but this instance is stuck.

If I try to terminate the instance again, nova-api returns the following warning:
2011-10-21 13:09:05,286 WARNING nova.compute.api [-] Instance 1145 is not in an 'active' state. It is currently deleted. Shutdown aborted.

The instance's disk image no longer exists on the compute node's filesystem.

[Thierry Carrez (ttx)]

Right, I guess there should be cleanups to prevent data inconsistency over the long run.

[Thierry Carrez (ttx)]

Looks like zombie undead instances also count against your quota, as stated in duplicate bug 862475

[Vish Ishaya (vishvananda)]

can you verify that this still happens against current trunk? I believe zombie instances can be terminated now.

[Thierry Carrez (ttx)]

Any chance you could confirm the fix ?

[Andrew Glen-Young (aglenyoung)]

@Thierry: I believe that this issue is resolved/fixed.

[Thierry Carrez (ttx)]

OK will mark it invalid since I can't point to the commitID that would have fixed it. Thanks!