Ubuntu 14.04
Rebooting nova-compute node results in a non-running nova-compute service. This is due the fact that nova-compute starts before libvirt:
2014-10-19 01:49:20.686 2993 AUDIT nova.service [-] Starting compute node (version 2014.1.2) 2014-10-19 01:50:15.141 2993 WARNING nova.virt.libvirt.driver [-] URI qemu:///system does not support events: Cannot write data: Broken pipe 2014-10-19 01:50:15.147 2993 ERROR nova.openstack.common.threadgroup [-] internal error: client socket is closed 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup Traceback (most recent call last): 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/threadgroup.py", line 117, in wait 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup x.wait() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/threadgroup.py", line 49, in wait 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup return self.thread.wait() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup return self._exit_event.wait() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup return self.greenlet.switch() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/openstack/common/service.py", line 483, in run_servic e 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup service.start() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/service.py", line 163, in start 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup self.manager.init_host() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1030, in init_host 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup self.driver.init_host(host=self.host) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 658, in init_host 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup self._do_quality_warnings() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 641, in _do_quality_warnings 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2870, in get_host_capabilities 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup xmlstr = self._conn.getCapabilities() 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3303, in getCapabilities 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectGetCapabilities() failed', conn=self) 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup libvirtError: internal error: client socket is closed 2014-10-19 01:50:15.147 2993 TRACE nova.openstack.common.threadgroup
Ubuntu 14.04
Rebooting nova-compute node results in a non-running nova-compute service. This is due the fact that nova-compute starts before libvirt:
2014-10-19 01:49:20.686 2993 AUDIT nova.service [-] Starting compute node (version 2014.1.2) libvirt. driver [-] URI qemu:///system does not support events: Cannot write data: Broken pipe common. threadgroup [-] internal error: client socket is closed common. threadgroup Traceback (most recent call last): common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/openstack/ common/ threadgroup. py", line 117, in wait common. threadgroup x.wait() common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/openstack/ common/ threadgroup. py", line 49, in wait common. threadgroup return self.thread.wait() common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ greenthread. py", line 168, in wait common. threadgroup return self._exit_ event.wait( ) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ event.py" , line 116, in wait common. threadgroup return hubs.get_ hub().switch( ) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ hubs/hub. py", line 187, in switch common. threadgroup return self.greenlet. switch( ) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ greenthread. py", line 194, in main common. threadgroup result = function(*args, **kwargs) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/openstack/ common/ service. py", line 483, in run_servic common. threadgroup service.start() common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/service. py", line 163, in start common. threadgroup self.manager. init_host( ) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/compute/ manager. py", line 1030, in init_host common. threadgroup self.driver. init_host( host=self. host) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/virt/ libvirt/ driver. py", line 658, in init_host common. threadgroup self._do_ quality_ warnings( ) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/virt/ libvirt/ driver. py", line 641, in _do_quality_ warnings common. threadgroup caps = self.get_ host_capabiliti es() common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ nova/virt/ libvirt/ driver. py", line 2870, in get_host_ capabilities common. threadgroup xmlstr = self._conn. getCapabilities () common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ tpool.py" , line 179, in doit common. threadgroup result = proxy_call( self._autowrap, f, *args, **kwargs) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ tpool.py" , line 139, in proxy_call common. threadgroup rv = execute( f,*args, **kwargs) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ eventlet/ tpool.py" , line 77, in tworker common. threadgroup rv = meth(*args, **kwargs) common. threadgroup File "/usr/lib/ python2. 7/dist- packages/ libvirt. py", line 3303, in getCapabilities common. threadgroup if ret is None: raise libvirtError ('virConnectGet Capabilities( ) failed', conn=self) common. threadgroup libvirtError: internal error: client socket is closed common. threadgroup
2014-10-19 01:50:15.141 2993 WARNING nova.virt.
2014-10-19 01:50:15.147 2993 ERROR nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
e
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.
2014-10-19 01:50:15.147 2993 TRACE nova.openstack.