Comment 9 for bug 1801897

Reviewed: https://review.opendev.org/656382
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c280d747fb23f1abaaf91eea7f6d11e716c6db42
Submitter: Zuul
Branch: stable/stein

commit c280d747fb23f1abaaf91eea7f6d11e716c6db42
Author: Andrey Volkov <email address hidden>
Date: Thu Feb 14 15:39:45 2019 +0300

    AZ list performance optimization: avoid double service list DB fetch

    Assume number of services can be large (10000 as in the bug description),
    this patch removes second service_get_all call.

    zone_hosts changed from dict of lists to dict of sets.

    The HostAPI instance from the API controller is also passed to the
    get_availability_zones method so it does not have to recreate it
    per call (this is both for a slight performance gain but mostly also
    for test sanity).

    On devstack with 10000 services patch decreased response time twice.

    openstack availability zone list --long --timing

    ...

    Before:

    +-------------------------------------------------------------------------------------------+-----------+
    | URL | Seconds |
    +-------------------------------------------------------------------------------------------+-----------+
    | GET http://192.168.0.45/identity | 0.006816 |
    | POST http://192.168.0.45/identity/v3/auth/tokens | 0.456708 |
    | POST http://192.168.0.45/identity/v3/auth/tokens | 0.087485 |
    | GET http://172.18.237.203/compute/v2.1/os-availability-zone/detail | 95.667192 |
    | GET http://172.18.237.203/volume/v2/e2671d37ee2c4374bd1533645261f1d4/os-availability-zone | 0.036528 |
    | Total | 96.254729 |
    +-------------------------------------------------------------------------------------------+-----------+

    After:

    +-------------------------------------------------------------------------------------------+-----------+
    | URL | Seconds |
    +-------------------------------------------------------------------------------------------+-----------+
    | GET http://192.168.0.45/identity | 0.020215 |
    | POST http://192.168.0.45/identity/v3/auth/tokens | 0.102987 |
    | POST http://192.168.0.45/identity/v3/auth/tokens | 0.111899 |
    | GET http://172.18.237.203/compute/v2.1/os-availability-zone/detail | 39.346657 |
    | GET http://172.18.237.203/volume/v2/e2671d37ee2c4374bd1533645261f1d4/os-availability-zone | 0.026403 |
    | Total | 39.608161 |
    +-------------------------------------------------------------------------------------------+-----------+

    The test_availability_zone_detail unit test is updated to assert that
    services are only retrieved twice (once for enabled, once for disabled).
    While in there, the expected response dict is formatted for readability
    and a duplicate zone/host is added to make sure duplicates are handled
    for available services. To ensure service_get_all is called only twice,
    the low-level DB API service_get_all stub is replaced with a mock and
    the mock is changed to be on the HostAPI.service_get_all method which
    is (1) what the API controller code is actually using and (2) allows the
    test to only mock the instance of the HostAPI being tested - trying to
    mock the DB API service_get_all method causes intermittent failures
    in unrelated tests because of the global nature of that mock.

    There is another opportunity for optimizing get_availability_zones which
    is marked with a TODO but left for a separate patch.

    Co-Authored-By: Matt Riedemann <email address hidden>

    Partial-Bug: #1801897
    Change-Id: Ib9a9a9a79499272d740a64cc0b909f0299a237d1
    (cherry picked from commit 74cefe4266a613d4c2afbb0c791e16eb7789aef4)