Nova compute starts too early when combined with libvirt and Xen

[Vangelis Tasoulas]

I have a nova-compute/libvirt/xen environment (which is not working properly yet due to another bug I reported here: https://bugs.launchpad.net/nova/+bug/1190262) and after each reboot nova-compute is not starting. The log shows that it tries to connect to localhost:8000 but it fails. Obviously Xen hasn't come up yet before nova.

2013-06-12 17:36:21.991 1191 INFO nova.manager [-] Skipping periodic task _periodic_update_dns because its interval is negative
2013-06-12 17:36:22.965 1191 INFO nova.virt.driver [-] Loading compute driver 'libvirt.LibvirtDriver'
2013-06-12 17:36:23.743 INFO nova.openstack.common.rpc.common [req-382618f3-28af-4452-8189-1d0ef7de1aaf None None] Connected to AMQP server on 10.0.0.1:5672
2013-06-12 17:36:23.873 1191 AUDIT nova.service [-] Starting compute node (version 2013.1.1)
2013-06-12 17:36:23.937 1191 ERROR nova.virt.libvirt.driver [-] Connection to libvirt failed: unable to connect to 'localhost:8000': Connection refused
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver Traceback (most recent call last):
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 627, in _connect
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver return libvirt.openAuth(uri, auth, 0)
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/libvirt.py", line 102, in openAuth
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver libvirtError: unable to connect to 'localhost:8000': Connection refused
2013-06-12 17:36:23.937 1191 TRACE nova.virt.libvirt.driver
2013-06-12 17:36:24.012 1191 WARNING nova.virt.libvirt.driver [-] URI xen:/// does not support events
2013-06-12 17:36:24.015 1191 ERROR nova.virt.libvirt.driver [-] Nova requires libvirt version 0.9.6 or greater.
2013-06-12 17:36:24.066 1191 ERROR nova.virt.libvirt.driver [-] Connection to libvirt failed: unable to connect to 'localhost:8000': Connection refused
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver Traceback (most recent call last):
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 627, in _connect
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver return libvirt.openAuth(uri, auth, 0)
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/dist-packages/libvirt.py", line 102, in openAuth
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver libvirtError: unable to connect to 'localhost:8000': Connection refused
2013-06-12 17:36:24.066 1191 TRACE nova.virt.libvirt.driver
2013-06-12 17:36:24.081 WARNING nova.virt.libvirt.driver [req-a35b8394-98fa-4bdd-9a26-82fe9104bfbf None None] URI xen:/// does not support events
2013-06-12 17:36:24.102 1191 CRITICAL nova [-] 'NoneType' object has no attribute 'numOfDomains'
2013-06-12 17:36:24.102 1191 TRACE nova Traceback (most recent call last):
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/bin/nova-compute", line 85, in <module>
2013-06-12 17:36:24.102 1191 TRACE nova service.wait()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 689, in wait
2013-06-12 17:36:24.102 1191 TRACE nova _launcher.wait()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 209, in wait
2013-06-12 17:36:24.102 1191 TRACE nova super(ServiceLauncher, self).wait()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 179, in wait
2013-06-12 17:36:24.102 1191 TRACE nova service.wait()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait
2013-06-12 17:36:24.102 1191 TRACE nova return self._exit_event.wait()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2013-06-12 17:36:24.102 1191 TRACE nova return hubs.get_hub().switch()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2013-06-12 17:36:24.102 1191 TRACE nova return self.greenlet.switch()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main
2013-06-12 17:36:24.102 1191 TRACE nova result = function(*args, **kwargs)
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 147, in run_server
2013-06-12 17:36:24.102 1191 TRACE nova server.start()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 429, in start
2013-06-12 17:36:24.102 1191 TRACE nova self.manager.init_host()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 597, in init_host
2013-06-12 17:36:24.102 1191 TRACE nova self._destroy_evacuated_instances(context)
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 434, in _destroy_evacuated_instances
2013-06-12 17:36:24.102 1191 TRACE nova local_instances = self._get_instances_on_driver(context)
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 397, in _get_instances_on_driver
2013-06-12 17:36:24.102 1191 TRACE nova driver_uuids = self.driver.list_instance_uuids()
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 681, in list_instance_uuids
2013-06-12 17:36:24.102 1191 TRACE nova for name in self.list_instances()]
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 663, in list_instances
2013-06-12 17:36:24.102 1191 TRACE nova for domain_id in self.list_instance_ids():
2013-06-12 17:36:24.102 1191 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 657, in list_instance_ids
2013-06-12 17:36:24.102 1191 TRACE nova if self._conn.numOfDomains() == 0:
2013-06-12 17:36:24.102 1191 TRACE nova AttributeError: 'NoneType' object has no attribute 'numOfDomains'
2013-06-12 17:36:24.102 1191 TRACE nova

[Launchpad Janitor (janitor)]

This bug was fixed in the package nova - 1:2014.1~b2-0ubuntu3

---------------
nova (1:2014.1~b2-0ubuntu3) trusty; urgency=medium

  * debian/tests/nova-compute-daemons: Drop nova-compute-uml.
 -- Chuck Short <email address hidden> Fri, 31 Jan 2014 14:40:23 -0500

[Brian Murray (brian-murray)]

Hello Vangelis, or anyone else affected,

Accepted nova into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/1:2013.2.2-0ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Dave Walker (davewalker)]

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.