eucalyptus-nc service fail to start at boot, cannot connect to libvirtd

[Etienne Goyer]

Binary package hint: eucalyptus-nc

At boot, eucalyptus-nc service start-up spit out the following on the console:

Node Controller cannot be started: errors in /var/log/eucalytpus/euca_test_nc.log

/var/log/eucalytpus/euca_test_nc.log contains the following:

-------------------------------------------------------------------------------------------------------
ubuntu@canonical-nc:~$ cat /var/log/eucalyptus/euca_test_nc.log

This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

total_memory=3738
nr_cores=2
libvir: Remote error : unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory
libvirt error: unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory (code=38)
error: failed to connect to hypervisor
-------------------------------------------------------------------------------------------------------

It sounds like a timing issue. I see that all three services eucalyptus-nc, libvirtd and kvm starts at S20. I tried changing eucalyptus-nc to S25, but got the same error. Not sure how that would need to be resolved.

This was on Jaunty (obviously!) amd64, up-to-date as of today.

[Soren Hansen (soren)]

I presume restarting the node controller fixes it?

Starting eucalyptus at S20 is too early anyway. It really should be around S90 or something.

[Etienne Goyer (etienne-goyer-outlands)]

Yes indeed, /etc/init.d/eucalyptus-nc (re)start once the node controller is done booting work as expected.

[Nilesh (lvnilesh)]

at boot, jaunty server shows

Node Controller cannot be started: errors in /var/log/eucalytpus/euca_test_nc.log

my copy of /var/log/eucalytpus/euca_test_nc.log is below:

nilesh@asterisk:~$ cat /var/log/eucalyptus/euca_test_nc.log

This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

total_memory=3917
nr_cores=2
libvir: Remote error : unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory
libvirt error: unable to connect to '/var/run/libvirt/libvirt-sock': No such file or directory (code=38)
error: failed to connect to hypervisor
nilesh@asterisk:~$

[Nilesh (lvnilesh)]

Then I restart as below.

nilesh@asterisk:~$ sudo /etc/init.d/eucalyptus-nc restart
 * Restarting Eucalyptus Node Controller
Warning! Cannot find bridge xenbr0: instances may be without net

And look at the log again which show no issues this time.

nilesh@asterisk:~$ cat /var/log/eucalyptus/euca_test_nc.log

This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2007, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

total_memory=3917
nr_cores=2

[Torsten Spindler (tspindler)]

Changing the start order will have eucalyptus-nc start after a reboot:

$ sudo update-rc.d -f eucalyptus-nc remove
$ sudo update-rc.d eucalyptus-nc start 99 1 2 3 4 5 . stop 99 0 6 .

[Etienne Goyer (etienne-goyer-outlands)]

Nilesh, as of your second comment, you need to change the value of VNET_BRIDGE in /etc/eucalyptus/eucalyptus.conf on the node controllers.

[Etienne Goyer (etienne-goyer-outlands)]

This bug have been fixed in karmic. It should be SRUed for jaunty, or marked WONTFIX, as appropriate.

[Dustin Kirkland  (kirkland)]

Per Etienne, marking fix released in Karmic.

I'll leave it open against Jaunty for now, in case an SRU is necessary, though I'd expect this is probably a "won't fix", please upgrade to Karmic situation.

:-Dustin

[Clint Byrum (clint-fewbar)]

Since Jaunty is EOL, closing Jaunty task as Won't Fix.