unexpected errors after a sequence of tests

Bug #622818 reported by C de-Avillez on 2010-08-23
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released
chris grzegorczyk
eucalyptus (Ubuntu)
Dave Walker
Dave Walker

Bug Description

euca2ools 1.2-0ubuntu10 install ok installed
eucalyptus-cloud 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
eucalyptus-common 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
eucalyptus-java-common 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
libeucalyptus-commons-ext-java 0.5.0-0ubuntu2 eucalyptus-commons-ext install ok installed
uec-component-listener 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed

During this weekend I ran a series of 500-instances tests. I decided on 500 because this makes the tests reasonably short (a few hours at most). Every so often, I would return, have a look and, if the run had finished, I would start a new one.

This morning, when I looked at the terminal, I saw many instance start failures. I then cancelled the test run, and ran a -describe-instances to find out what was running. It failed:

ubuntu@cempedak:/var/log/eucalyptus$ euca-describe-instances --debug
[Errno 104] Connection reset by peer

Experiments showed that no euca-* would return.

I have uploaded the logs to http://bazaar.launchpad.net/~hggdh2/+junk/uec-qa/files at revision 47.

Mathias Gug (mathiaz) on 2010-08-25
Changed in eucalyptus (Ubuntu):
importance: Undecided → Medium
Changed in eucalyptus:
assignee: nobody → chris grzegorczyk (chris-grze)
Dmitrii Zagorodnov (dmitrii) wrote :

Carlos, when you get a chance, would you try to capture the logs again with " --debug --log-level=DEBUG" appended to CLOUD_OPTS in eucalyptus.conf on the CLC machine? Thank you!

C de-Avillez (hggdh2) wrote :

Dmitrii,, done: lp:~hggdh2/+junk/uec-qa:

Using saved push location: bzr+ssh://bazaar.launchpad.net/~hggdh2/%2Bjunk/uec-qa/
Pushed up to revision 50.

Seems we starting having problems at around 20:00 (or earlier, still parsing thru the logs). I noticed it at around 21:30ish, and cancelled the test run (and left it resting until this morning).

Changed in eucalyptus:
status: New → Fix Committed
chris grzegorczyk (chris-grze) wrote :

Hi Carlos,

I committed something that was lost in a merge at some point and should address this issue. Let me know how it goes when you have a chance to try reproducing. Thanks.

revno: 1238 [merge]
committer: decker <decker@hawaii>
branch nick: 2.0.0-lp
timestamp: Mon 2010-09-06 19:45:17 -0700
  clean up in metadata pipeline; restore fix lost in merge. LP: #622818

Thierry Carrez (ttx) on 2010-09-07
Changed in eucalyptus (Ubuntu Maverick):
importance: Medium → High
tags: added: server-mrs
Thierry Carrez (ttx) wrote :

Needs to be rolled out in a package now

Changed in eucalyptus (Ubuntu Maverick):
assignee: nobody → Dave Walker (davewalker)
status: New → Triaged
Dave Walker (davewalker) on 2010-09-10
Changed in eucalyptus (Ubuntu Maverick):
status: Triaged → In Progress
status: In Progress → Fix Committed
Thierry Carrez (ttx) wrote :

eucalyptus (2.0+bzr1239-0ubuntu1) maverick; urgency=low

  [Dave Walker (Daviey)]
  * New upstream stable release, -r1239 Fixes:
    - euca_conf --register-cluster is not idempotent. (LP: #628328)
    - unexpected errors after a sequence of tests (LP: #622818)
  * debian/control:
    - Recommend bittorrent|bittornado for -walrus package, enabling S3
      bittorent API support. (LP: #613535)
    - Raise the minimum version of libvirt-bin needed to eucalyptus-nc.
  * debian/eucalyptus-nc.postinst: Add eucalyptus user to the kvm group to
    enable user to utilise console logs. (LP: #619843)
  * debian/patches/18-priv_security.patch: Added vgscan to util/wrappers.conf,
    as it requires escalated privileges. (LP: #628291)
  * debian/patches/06-UEC-webinterface.patch (and 3 images): Updated and
    refreshed to better reflect Ubuntu branding. (LP: #631451)
  * debian/patches/25-uec-logo-png.patch: Due to a bug in dpkg, we need to use
    a different filename for the logo.png.
  * debian/eucalyptus-{java-,}common.{preinst,postinst}: Update the internal
    database and keys on upgrade from 10.04.

  [Scott Moser]
  * fix determination of 'is_multiboot'.
 -- Dave Walker (Daviey) <email address hidden> Fri, 10 Sep 2010 12:18:02 +0100

Changed in eucalyptus (Ubuntu Maverick):
status: Fix Committed → Fix Released
Changed in eucalyptus:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers