reduce logging in the scheduler to improve performance

[John Garbutt]

The current debug logs in the scheduler are at critical points in the code, and are causing performance issues.

After the DB, the scheduler is spending more time doing logging, than anything else.

This was discovered using the test_performance_check_select_destination unit test, and modifying it to look at when there are around 200 hosts, which is still quite a modest size.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/84806

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/84806
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=93f29574c416aebcd1bd7527d3007664aa073766
Submitter: Jenkins
Branch: master

commit 93f29574c416aebcd1bd7527d3007664aa073766
Author: John Garbutt <email address hidden>
Date: Wed Apr 2 17:37:03 2014 +0100

    Reduce logging in scheduler

    The logging in the scheduler is what takes most of the time,
    once you are doing no DB calls.

    This ensures the servicegroup is_up and retry_filter only log
    on failures, which is more in line with the other filters.

    Using the test "test_performance_check_select_destination"
    modifying it to use 200 hosts, the request time went from
    70ms to under 10ms.

    We could reduce logging further, but this minimal change makes
    a massive impact. The next steps would require a config value
    to disable all debug logging in the scheduler.

    Closes-Bug: #1301515

    Change-Id: Ia4c7e506e8df9c560c349e2ebec3f5111aa0cb0d