"Ghost" instance using a slot + elastic IP

Bug #676832 reported by Kiall Mac Innes on 2010-11-18
2
Affects Status Importance Assigned to Milestone
Eucalyptus
Confirmed
Low
Unassigned
2.0
Confirmed
Low
Unassigned
eucalyptus (Ubuntu)
Low
Unassigned

Bug Description

With a single instance running, 2 instance slots are used up and an elastic IP is in-use by a "ghost" instance..

I can confirm that there is only a single kvm process on the NC, and eucalyptus seems to know nothing about the instance other than its reserved an IP.

Other than this, I really have no info on how this occurred. I'm really just filing this here so its logged!

Also - I have the NC logs saved - but there pretty large (20MB total). I'll attach if LP lets me!

kiall@wk08-lmst:~/Desktop/CloudConfig$ euca-describe-availability-zones verbose
AVAILABILITYZONE lmst01 10.2.1.4
AVAILABILITYZONE |- vm types free / max cpu ram disk
AVAILABILITYZONE |- m1.small 0021 / 0023 1 512 5
AVAILABILITYZONE |- c1.medium 0005 / 0006 2 1740 5
AVAILABILITYZONE |- m1.large 0000 / 0001 2 7680 5
AVAILABILITYZONE |- m1.xlarge 0000 / 0000 4 15360 5
AVAILABILITYZONE |- c1.xlarge 0000 / 0000 8 15361 5

kiall@wk08-lmst:~/Desktop/CloudConfig$ euca-describe-instances
RESERVATION r-371406C3 kiall phostr-app
INSTANCE i-456F080E emi-077E15DD euca-172-19-12-66.eucalyptus.euca.managedit.ie euca-172-19-12-66.eucalyptus.internal running kiall 0

kiall@wk08-lmst:~/Desktop/CloudConfig$ euca-describe-addresses
... SNIP ...
ADDRESS 10.2.6.108 i-307905AD (kiall)
... SNIP ...

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: eucalyptus-cloud 2.0+bzr1241-0ubuntu4
ProcVersionSignature: Ubuntu 2.6.35-22.35-server 2.6.35.4
Uname: Linux 2.6.35-22-server x86_64
.etc.eucalyptus.eucalyptus.cc.conf: CC_NAME="lmst01"
Architecture: amd64
Date: Thu Nov 18 06:22:18 2010
InstallationMedia: Ubuntu-Server 10.10 "Maverick Meerkat" - Release amd64 (20101007)
ProcEnviron:
 LANG=en_IE.UTF-8
 SHELL=/bin/bash
SourcePackage: eucalyptus

Kiall Mac Innes (kiall) wrote :
Kiall Mac Innes (kiall) wrote :
Kiall Mac Innes (kiall) wrote :
Kiall Mac Innes (kiall) wrote :
Kiall Mac Innes (kiall) wrote :

`service eucalyptus-(nc|walrus|sc|cloud|cc) restart` each had no effect...

`service eucalyptus restart` free'd the IP..

Dave Walker (davewalker) wrote :

Interesting, thanks for raising this bug. I have looked at the content of the logs, and nothing is obviously at fault. I'm marking the Ubuntu task as incomplete, and will discuss the issue with the project upstream to see if they can add any more insight.

Thanks.

Changed in eucalyptus (Ubuntu):
status: New → Incomplete
Changed in eucalyptus:
assignee: nobody → chris grzegorczyk (chris-grze)
Kiall Mac Innes (kiall) wrote :

I should have also mentioned - the "Ghost" instance (i-307905AD) was a real instance that had been terminated, possibly before it had left the "pending" state.

I think this is a known issue in 2.0.x where, under certain conditions, some IPs are still marked in use. It seems to be triggered by instance not starting properly (but not only).

Forgot to mention: the workaround is to cleanrestart the CC (one will need to stop all running instances).

Changed in eucalyptus:
assignee: chris grzegorczyk (chris-grze) → nobody
status: New → Confirmed
importance: Undecided → Low
James Page (james-page) on 2012-04-27
Changed in eucalyptus (Ubuntu):
importance: Undecided → Low
Andy Grimm (agrimm) wrote :

This issue is now being tracked upstream at http://eucalyptus.atlassian.net/browse/EUCA-2743

Please watch that issue for further updates.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers