libvirt.libvirtError: Domain not found: no domain with matching uuid

[sean mooney]

seen in upstream ci in grenade multi node job

as part of a livemigation.
in this canse the error happens on the destination host.

Traceback (most recent call last):

   File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 605, in _get_domain

     return conn.lookupByUUIDString(instance.uuid)

   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 190, in doit

     result = proxy_call(self._autowrap, f, *args, **kwargs)

   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 148, in proxy_call

     rv = execute(f, *args, **kwargs)

   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 129, in execute

     six.reraise(c, e, tb)

   File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise

     raise value

   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 83, in tworker

     rv = meth(*args, **kwargs)

   File "/usr/local/lib/python3.6/dist-packages/libvirt.py", line 4151, in lookupByUUIDString

     if ret is None:raise libvirtError('virDomainLookupByUUIDString() failed', conn=self)

 libvirt.libvirtError: Domain not found: no domain with matching uuid '386113bf-cca1-438a-9ab5-4714c147bbfc'

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):

   File "/opt/stack/old/nova/nova/compute/manager.py", line 8005, in _do_pre_live_migration_from_source

     instance, block_device_info=block_device_info)

   File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 9934, in get_instance_disk_info

     self._get_instance_disk_info(instance, block_device_info))

   File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 9915, in _get_instance_disk_info

     guest = self._host.get_guest(instance)

   File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 589, in get_guest

     return libvirt_guest.Guest(self._get_domain(instance))

   File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 609, in _get_domain

     raise exception.InstanceNotFound(instance_id=instance.uuid)

 nova.exception.InstanceNotFound: Instance 386113bf-cca1-438a-9ab5-4714c147bbfc could not be found.

this seam similar to
https://bugs.launchpad.net/nova/+bug/1662626 but its not really clear why this fails

see https://zuul.opendev.org/t/openstack/build/cfda29fa579544e481c803c4c5de51fb/log/logs/subnode-2/screen-n-cpu.txt#9697-9729
of https://zuul.opendev.org/t/openstack/build/cfda29fa579544e481c803c4c5de51fb/ for full logs

[do3meli (d-info-e)]

we just experienced the same in our production environment with ubuntu 20.04 running ussuri with the following package versions:

libvirt-clients 6.0.0-0ubuntu8.10
libvirt-daemon 6.0.0-0ubuntu8.10
libvirt-daemon-driver-qemu 6.0.0-0ubuntu8.10
libvirt-daemon-driver-storage-rbd 6.0.0-0ubuntu8.10
libvirt-daemon-system 6.0.0-0ubuntu8.10
libvirt-daemon-system-systemd 6.0.0-0ubuntu8.10
libvirt0:amd64 6.0.0-0ubuntu8.10
nova-compute-libvirt 2:21.2.0-0ubuntu1
python3-libvirt 6.1.0-1
nova-common 2:21.2.0-0ubuntu1
nova-compute 2:21.2.0-0ubuntu1
nova-compute-kvm 2:21.2.0-0ubuntu1
nova-compute-libvirt 2:21.2.0-0ubuntu1
python3-nova 2:21.2.0-0ubuntu1
python3-novaclient 2:17.0.0-0ubuntu1