Misleading ERROR message relating to instances not found in database

[David McNally]

We are seeing the following error message in the nova-compute logs:

Instance <ID> found in the hypervisor, but not in the database

Upon investigation we found that the instance is running on the hypervisor and correctly entered in the database.

I've tracked this down to a function in the compute manager _get_instances_on_driver. This method is called periodically from
 _running_deleted_instances (and possibly from other places). _get_instances_on_driver calls instance_get_all_by_filters and, in this case, the filters are set to retrieve deleted instances on the current host. The list of instances retrieved is compared to the list of instances currently running on the hypervisor. Any instances on the host but not listed in the return from the db query are logged in the above error format, however this means instances which are perfectly healthy are getting logged simply because they are not set to deleted.

This results in an error message for every instance on the host being logged each time the periodic task happens. I think we need to remove the log message in this case as it can't know what filters have been specified and so can't provide useful information and just causes noise for anyone monitoring errors.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/30270

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/30270
Committed: http://github.com/openstack/nova/commit/28b2deac2f9d835cc24d8b5c015762a9780df99b
Submitter: Jenkins
Branch: master

commit 28b2deac2f9d835cc24d8b5c015762a9780df99b
Author: David McNally <email address hidden>
Date: Thu May 23 14:13:10 2013 +0100

    Removing misleading error message

    This function (_get_instances_on_driver) was reporting instances are on the
    hypervisor and not in the database with no consideration for the filters being
    used in the database query. This resulted in error log messages indicating running
    instances were not in the database when in fact the query was filtering for deleted
    instances.

    To avoid misleading error logs of this nature I've removed the logging from the
    function. Currently the locations which call this function provide more accurate
    logging on the state of the instances that are returned based on the filters they
    have called _get_instances_on_driver with.

    bug 1183276

    Change-Id: I4ee52fe417f73a90cf157bc531b483be93e34275