With the above patch, instances are still not launching, returning this traceback [1] so this cgroup or namespace change is also affecting nova_libvirtd presumably.
[1]
~~~
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [req-b526d04d-1bfe-49e3-8a9d-19f6e93225e4 aa9659a889c3439485a7a307256e4250 4a886606e5a44e9e838582c33347166d - default default] [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] Instance failed to spawn: libvirt.libvirtError: unable to open '/sys/fs/cgroup/machine/qemu-1-instance-00000002.libvirt-qemu/': No such file or directory
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] Traceback (most recent call last):
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2640, in _build_resources
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] yield resources
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2413, in _build_and_run_instance
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] accel_info=accel_info)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4193, in spawn
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] cleanup_instance_disks=created_disks)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7250, in _create_guest_with_network
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] cleanup_instance_disks=cleanup_instance_disks)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] self.force_reraise()
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] raise self.value
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7218, in _create_guest_with_network
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] post_xml_callback=post_xml_callback)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7155, in _create_guest
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] guest.launch(pause=pause)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 163, in launch
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] self._encoded_xml, errors='ignore')
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] self.force_reraise()
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] raise self.value
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 159, in launch
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] return self._domain.createWithFlags(flags)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] result = proxy_call(self._autowrap, f, *args, **kwargs)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] rv = execute(f, *args, **kwargs)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] six.reraise(c, e, tb)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] raise value
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 83, in tworker
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] rv = meth(*args, **kwargs)
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] File "/usr/lib64/python3.6/site-packages/libvirt.py", line 1385, in createWithFlags
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] raise libvirtError('virDomainCreateWithFlags() failed')
nova-compute.log:2021-07-14 00:15:23.206 2 ERROR nova.compute.manager [instance: 347d7250-b14a-4bfb-8aab-7a3580847865] libvirt.libvirtError: unable to open '/sys/fs/cgroup/machine/qemu-1-instance-00000002.libvirt-qemu/': No such file or directory
~~~
With the above patch, instances are still not launching, returning this traceback [1] so this cgroup or namespace change is also affecting nova_libvirtd presumably.
[1] log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [req-b526d04d- 1bfe-49e3- 8a9d-19f6e93225 e4 aa9659a889c3439 485a7a307256e42 50 4a886606e5a44e9 e838582c3334716 6d - default default] [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] Instance failed to spawn: libvirt. libvirtError: unable to open '/sys/fs/ cgroup/ machine/ qemu-1- instance- 00000002. libvirt- qemu/': No such file or directory log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] Traceback (most recent call last): log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/compute/ manager. py", line 2640, in _build_resources log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] yield resources log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/compute/ manager. py", line 2413, in _build_ and_run_ instance log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] accel_info= accel_info) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ driver. py", line 4193, in spawn log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] cleanup_ instance_ disks=created_ disks) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ driver. py", line 7250, in _create_ guest_with_ network log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] cleanup_ instance_ disks=cleanup_ instance_ disks) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ oslo_utils/ excutils. py", line 227, in __exit__ log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] self.force_ reraise( ) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ oslo_utils/ excutils. py", line 200, in force_reraise log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] raise self.value log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ driver. py", line 7218, in _create_ guest_with_ network log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] post_xml_ callback= post_xml_ callback) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ driver. py", line 7155, in _create_guest log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] guest.launch( pause=pause) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ guest.py" , line 163, in launch log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] self._encoded_xml, errors='ignore') log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ oslo_utils/ excutils. py", line 227, in __exit__ log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] self.force_ reraise( ) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ oslo_utils/ excutils. py", line 200, in force_reraise log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] raise self.value log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ nova/virt/ libvirt/ guest.py" , line 159, in launch log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] return self._domain. createWithFlags (flags) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ eventlet/ tpool.py" , line 190, in doit log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] result = proxy_call( self._autowrap, f, *args, **kwargs) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ eventlet/ tpool.py" , line 148, in proxy_call log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] rv = execute(f, *args, **kwargs) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ eventlet/ tpool.py" , line 129, in execute log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] six.reraise(c, e, tb) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ six.py" , line 703, in reraise log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] raise value log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib/ python3. 6/site- packages/ eventlet/ tpool.py" , line 83, in tworker log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] rv = meth(*args, **kwargs) log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] File "/usr/lib64/ python3. 6/site- packages/ libvirt. py", line 1385, in createWithFlags log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] raise libvirtError( 'virDomainCreat eWithFlags( ) failed') log:2021- 07-14 00:15:23.206 2 ERROR nova.compute. manager [instance: 347d7250- b14a-4bfb- 8aab-7a35808478 65] libvirt. libvirtError: unable to open '/sys/fs/ cgroup/ machine/ qemu-1- instance- 00000002. libvirt- qemu/': No such file or directory
~~~
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
nova-compute.
~~~