Comment 1 for bug 1600031

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22_get_disk_over_committed_size_total%5C%22%20AND%20message%3A%5C%22local_instances%5Bguest.uuid%5D%2C%20bdms%5Bguest.uuid%5D)%5C%22%20AND%20message%3A%5C%22KeyError%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d

Shows this is new in the last 24 hours, so we should look for anything in nova in the last 24 hours related to BDMs or deletion code because looking at the code, it's getting all instances from the host (via libvirt), pulling those from the DB, and then pulling the BDMs from the DB for those instances, and if we're racing with delete, those might not match.

instance uuids from the host:

https://github.com/openstack/nova/blob/d2a89a85354ab7253d034335aa2eb996e81b6f1c/nova/virt/libvirt/driver.py#L6803

and use those to get them from the DB:

https://github.com/openstack/nova/blob/d2a89a85354ab7253d034335aa2eb996e81b6f1c/nova/virt/libvirt/driver.py#L6815

Which includes deleted instances by default.

Then use that instance uuid list to get BDMs:

https://github.com/openstack/nova/blob/d2a89a85354ab7253d034335aa2eb996e81b6f1c/nova/virt/libvirt/driver.py#L6821

Then get the guest object and check to see if it's in the list of instances from the database, which it is:

https://github.com/openstack/nova/blob/d2a89a85354ab7253d034335aa2eb996e81b6f1c/nova/virt/libvirt/driver.py#L6830

But it's not in the BDMs list, maybe because the BDM is deleted already.