The node controller sometimes fails to start if libvirt is not ready

Bug #446036 reported by Mathias Gug
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
eucalyptus (Ubuntu)
Fix Released
Medium
Dustin Kirkland 
libvirt (Ubuntu)
Fix Released
Medium
Dustin Kirkland 

Bug Description

While rebooting node controllers multiple times, the nc fails sometimes to start. It seems related to libvirt not being ready as noted in /var/log/eucalyptus/euca_test_nc.log:

ubuntu@uec-nc4:~$ cat /var/log/eucalyptus/euca_test_nc.log

This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

total_memory=891
nr_cores=2
libvir: Remote error : unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory
libvirt error: unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory (code=38)
error: failed to connect to hypervisor
ubuntu@uec-nc4:~$

Restarting eucalyptus-nc fixes the problem: the ressource of the node are correctly registered with the CC.

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Ideally, we'd have an upstart libvirt configuration, and we'd solve Bug #438631.

Then, we could ensure that the necessary resources are available.

:-Dustin

Changed in eucalyptus (Ubuntu):
importance: Undecided → Medium
status: New → Confirmed
Thierry Carrez (ttx)
Changed in eucalyptus (Ubuntu):
status: Confirmed → Triaged
Thierry Carrez (ttx)
Changed in eucalyptus (Ubuntu):
importance: Medium → Low
Revision history for this message
Matt Zimmerman (mdz) wrote :

Dan says this is the same root cause as bug 452572

Changed in eucalyptus (Ubuntu):
importance: Low → Medium
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

This could/should be solved with a libvirt upstart script.

I'm working on it.

Changed in libvirt (Ubuntu):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Dustin Kirkland (kirkland)
Changed in eucalyptus (Ubuntu):
assignee: nobody → Dustin Kirkland (kirkland)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libvirt - 0.7.2-4ubuntu5

---------------
libvirt (0.7.2-4ubuntu5) lucid; urgency=low

  * debian/rules, debian/libvirt-bin.upstart: migrate libvirt to upstart
    (LP: #446036)
  * debian/control: depend on open-iscsi-utils, rather than open-iscsi,
    fixed again, grrr (LP: #414986)
 -- Dustin Kirkland <email address hidden> Tue, 05 Jan 2010 23:23:28 -0600

Changed in libvirt (Ubuntu):
status: In Progress → Fix Released
Changed in eucalyptus (Ubuntu):
status: Triaged → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package eucalyptus - 1.6.2~bzr1120-0ubuntu4

---------------
eucalyptus (1.6.2~bzr1120-0ubuntu4) lucid; urgency=low

  [ Thierry Carrez ]
  * debian/*publication.upstart: Start publication jobs when eth0 is up, and
    never stop them to work around The Upstart Bug (LP: #503850)

  [ Dustin Kirkland ]
  * debian/control, debian/eucalyptus-nc.upstart: (LP: #446036, #452572)
    - add a versioned depends for eucalyptus-nc on a new version
      of libvirt-bin that starts using upstart
    - start eucalyptus-nc on started libvirt-bin
 -- Dustin Kirkland <email address hidden> Wed, 06 Jan 2010 19:16:01 -0600

Changed in eucalyptus (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.