I installed openstack (Newton) on centos 7 using packstack all-in-one. When i launch an instance it goes to error state. Nova-compute logs are following:
2017-02-02 09:31:05.448 17380 INFO nova.virt.libvirt.driver [-] Connection event '0' reason 'Connection to libvirt lost: 0'
2017-02-02 09:31:05.456 17380 ERROR root [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch\n return self._domain.createWithFlags(flags)\n', ' File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call\n rv = execute(f, *args, **kwargs)\n', ' File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute\n six.reraise(c, e, tb)\n', ' File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker\n rv = meth(*args, **kwargs)\n', ' File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1065, in createWithFlags\n if ret == -1: raise libvirtError (\'virDomainCreateWithFlags() failed\', dom=self)\n', 'libvirtError: End of file while reading data: Input/output error\n']
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Instance failed to spawn
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Traceback (most recent call last):
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2078, in _build_resources
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] yield resources
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1920, in _build_and_run_instance
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] block_device_info=block_device_info)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2596, in spawn
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] post_xml_callback=gen_confdrive)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4836, in _create_domain_and_network
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] post_xml_callback=post_xml_callback)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4765, in _create_domain
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] guest.launch(pause=pause)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] self._encoded_xml, errors='ignore')
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 111, in _encoded_xml
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] return encodeutils.safe_decode(self._domain.XMLDesc(0))
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] rv = execute(f, *args, **kwargs)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] six.reraise(c, e, tb)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] rv = meth(*args, **kwargs)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 492, in XMLDesc
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] if ret is None: raise libvirtError ('virDomainGetXMLDesc() failed', dom=self)
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] libvirtError: internal error: client socket is closed
2017-02-02 09:31:05.457 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf]
2017-02-02 09:31:05.459 17380 INFO nova.compute.manager [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Terminating instance
2017-02-02 09:31:05.460 17380 WARNING nova.image.glance [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] No protocol specified in for api_server 'http://192.168.100.201:9292', please update [glance] api_servers with fully qualified url including scheme (http / https)
2017-02-02 09:31:05.508 17380 INFO nova.virt.libvirt.driver [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] Connection event '0' reason 'Failed to connect to libvirt'
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host Traceback (most recent call last):
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 411, in get_connection
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host conn = self._get_connection()
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 398, in _get_connection
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host wrapped_conn = self._get_new_connection()
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 350, in _get_new_connection
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host wrapped_conn = self._connect(self._uri, self._read_only)
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/host.py", line 204, in _connect
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host libvirt.openAuth, uri, auth, flags)
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host rv = execute(f, *args, **kwargs)
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host six.reraise(c, e, tb)
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host rv = meth(*args, **kwargs)
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host File "/usr/lib64/python2.7/site-packages/libvirt.py", line 105, in openAuth
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host if ret is None:raise libvirtError('virConnectOpenAuth() failed')
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2017-02-02 09:31:05.539 17380 ERROR nova.virt.libvirt.host
2017-02-02 09:31:05.547 17380 WARNING nova.compute.manager [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] Could not clean up failed build, not rescheduling. Error: Connection to the hypervisor is broken on host: compute1
2017-02-02 09:31:05.833 17380 WARNING nova.image.glance [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] No protocol specified in for api_server 'http://192.168.100.201:9292', please update [glance] api_servers with fully qualified url including scheme (http / https)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Build of instance 2b7c7015-4e13-4935-bdaf-4f0d090096bf aborted: internal error: client socket is closed
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Traceback (most recent call last):
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1779, in _do_build_and_run_instance
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] filter_properties)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1939, in _build_and_run_instance
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] 'create.error', fault=e)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] self.force_reraise()
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] six.reraise(self.type_, self.value, self.tb)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1923, in _build_and_run_instance
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] instance=instance)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] self.gen.throw(type, value, traceback)
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2105, in _build_resources
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] reason=six.text_type(exc))
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] BuildAbortException: Build of instance 2b7c7015-4e13-4935-bdaf-4f0d090096bf aborted: internal error: client socket is closed
2017-02-02 09:31:05.838 17380 ERROR nova.compute.manager [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf]
2017-02-02 09:31:06.732 17380 INFO nova.compute.manager [req-7430e29b-52b3-49f3-9587-9a7874bd27cc 9985770a1d4c49c980ee33f5dda4916f c64fdb67cbd54271ab9dc06201c5da54 - - -] [instance: 2b7c7015-4e13-4935-bdaf-4f0d090096bf] Took 0.89 seconds to deallocate network for instance.
It actually crashes libvirtd binary for a moment which breaks connection with nova and hence instance never gets created.
any update please?