ENOENT error breaks update_status() of libvirt driver

[Qin Zhao]

ENOENT error breaks update_status() during PowerKVM testing. It should be a bug of libvirt driver.

Exception log:

2014-08-15 16:03:59.038 42817 ERROR nova.openstack.common.periodic_task [-] Error during PowerVCComputeManager.update_available_resource: [Errno 2] No such file or directory: '/mnt//nova/instances/2132a870-ad5e-4d42-a43c-
ca1e93fe7eeb/disk'
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task Traceback (most recent call last):
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/openstack/common/periodic_task.py", line 198, in run_periodic_tasks
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task task(self, context)
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 5667, in update_available_resource
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task rt.update_available_resource(context)
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/powervc_nova/compute/manager.py", line 3805, in update_available_resource
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task self.driver.get_host_stats(refresh=True)
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5026, in get_host_stats
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task return self.host_state.get_host_stats(refresh=refresh)
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/powervc_nova/virt/powerkvm/driver.py", line 1688, in get_host_stats
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task self._data = self.update_status()
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/powervc_nova/virt/powerkvm/driver.py", line 1693, in update_status
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task data = super(PowerKVMHostState, self).update_status()
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5515, in update_status
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task data['disk_available_least'] = _get_disk_available_least()
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5488, in _get_disk_available_least
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task disk_over_committed = (self.driver.
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4984, in _get_disk_over_committed_size_total
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task self._get_instance_disk_info(dom.name(), xml))
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4937, in _get_instance_disk_info
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task dk_size = int(os.path.getsize(path))
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task File "/usr/lib64/python2.7/genericpath.py", line 49, in getsize
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task return os.stat(filename).st_size
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task OSError: [Errno 2] No such file or directory: '/mnt//nova/instances/2132a870-ad5e-4d42-a43c-ca1e93fe7eeb/disk'
2014-08-15 16:03:59.038 42817 TRACE nova.openstack.common.periodic_task

[Qin Zhao (zhaoqin)]

If errno is ENOENT, the code will print a warning and continue to raise the exception. I feel the correct behavior is not throwing this exception.

            except OSError as e:
                if e.errno == errno.ENOENT:
                    LOG.warn(_LW('Periodic task is updating the host stat, '
                                 'it is trying to get disk %(i_name)s, '
                                 'but disk file was removed by concurrent '
                                 'operations such as resize.'),
                                {'i_name': dom.name()})
                if e.errno == errno.EACCES:
                    LOG.warn(_LW('Periodic task is updating the host stat, '
                                 'it is trying to get disk %(i_name)s, '
                                 'but access is denied. It is most likely '
                                 'due to a VM that exists on the compute '
                                 'node but is not managed by Nova.'),
                             {'i_name': dom.name()})
                else:
                    raise

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/115194

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/115194
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3ca0817d1f78e8f63610b5d7667752f7f4a05b51
Submitter: Jenkins
Branch: master

commit 3ca0817d1f78e8f63610b5d7667752f7f4a05b51
Author: Qin Zhao <email address hidden>
Date: Tue Aug 19 14:56:12 2014 +0800

    Libvirt: Do not raise ENOENT exception

    If the disk disappears during update_status(), only need to print
    a warning, and do not need to raise an exception.

    Change-Id: I87800c855c7b349f69163e33031404ad25e2a9ca
    Closes-Bug: 1358609