After deployment finished, nova-compute have errors with not finding the '/var/run/libvirt/libvirt-sock'

Bug #1412293 reported by zhu zhu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack + Chef
Fix Released
Undecided
zhu zhu

Bug Description

With the success deployment of the nova-compute node, found from the compute logs.
After some time, the compute get started without any errors.

The root cause is that the nova-compute started up prior than the libvirtd services. So before the libvirtd sertice started up fully, nova-compute will try to attempt to connect to the Hypervisor and met the No Such File exception.

So in compute cookbook, it need to switch the order , install and startup the libvirtd process and then restart compute service to avoid any error logs in compute logs.

2015-01-17 00:07:08.499 13762 INFO nova.virt.libvirt.driver [-] Connection event '0' reason 'Failed to connect to libvirt'
2015-01-17 00:07:08.536 13762 WARNING nova.virt.libvirt.driver [req-22f54a55-bd8c-44ee-af7e-3195ba29f8ce None] Cannot update service status on host: test34.sce.ibm.com,since it is not registered.
2015-01-17 00:07:08.537 13762 ERROR nova.virt.libvirt.driver [-] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver Traceback (most recent call last):
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 548, in _get_connection
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver conn = self._host.get_connection()
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 380, in get_connection
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver wrapped_conn = self._get_new_connection()
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 332, in _get_new_connection
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver wrapped_conn = self._connect(self._uri, self._read_only)
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 183, in _connect
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver libvirt.openAuth, uri, auth, flags)
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver rv = execute(f, *args, **kwargs)
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver six.reraise(c, e, tb)
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver rv = meth(*args, **kwargs)
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver File "/usr/lib64/python2.7/site-packages/libvirt.py", line 102, in openAuth
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
2015-01-17 00:07:08.537 13762 TRACE nova.virt.libvirt.driver

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cookbook-openstack-compute (master)

Fix proposed to branch: master
Review: https://review.openstack.org/148195

Changed in openstack-chef:
assignee: nobody → zhu zhu (zhuzhubj)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cookbook-openstack-compute (master)

Reviewed: https://review.openstack.org/148195
Committed: https://git.openstack.org/cgit/stackforge/cookbook-openstack-compute/commit/?id=d85a24cba56661e54615de14725913afb28f7e3f
Submitter: Jenkins
Branch: master

commit d85a24cba56661e54615de14725913afb28f7e3f
Author: ZHU ZHU <email address hidden>
Date: Mon Jan 19 01:26:25 2015 -0600

    Fix compute service log error

    Compute service need to get start after libvirtd process get start.
    If not, compute process will raise Hypervisor connection error within
    initial attempts of compute resource updates due to lack of libvirt-sock
    file.

    Change-Id: I4ee0832e97ea93e60ba7978db54e439ace4631df
    Closes-Bug: #1412293

Changed in openstack-chef:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.