InstanceList.get_active_by_window_joined only needs system_metadata for expected_attrs in simple-tenant-usage API

[Matt Riedemann]

The os-simple-tenant-usage API queries the database over a given time range (default is 1 month at most from Horizon and novaclient) and pulls all instance information from the database for a given tenant.

As you add more instances, this API is going to take longer given the joins required in the database for the instances table here:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/simple_tenant_usage.py?id=2014.2#n142

However, the only join we really need there is on system_metadata when getting the flavor information:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/simple_tenant_usage.py?id=2014.2#n114

This means we're joining on these other tables unnecessarily:

INSTANCE_DEFAULT_FIELDS = ['metadata', 'system_metadata', 'info_cache', 'security_groups']

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/129732

[Lin Hua Cheng (lin-hua-cheng)]

I think we don't need to join with system_metadata either.

instance_type_id which is used in flavor (http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/simple_tenant_usage.py?id=2014.2#n114) is also available in the instance table.

[Lin Hua Cheng (lin-hua-cheng)]

The query also joins with info_cache and security_groups too, data from both tables are not used either.

https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2153

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/131250

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/131250
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=152767927c90eac83705a3847a1f538eecc386be
Submitter: Jenkins
Branch: master

commit 152767927c90eac83705a3847a1f538eecc386be
Author: Matt Riedemann <email address hidden>
Date: Mon Oct 27 12:17:32 2014 -0700

    DB API: Pass columns_to_join to instance_get_active_by_window_joined

    This makes the instance_get_active_by_window_joined database API
    configurable for which columns should be joined from other tables,
    similar to instance_get_all.

    This lays the groundwork for the Instance object to pass expected_attrs
    to the DB API for filtering which columns to load.

    Partial-Bug: #1383469

    Change-Id: I7089d732cbd06a854d47b7b9e4faf21b9ef498f1

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/131490

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/131490
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=203079caba3ba113009e6fcb0f8eea7247666761
Submitter: Jenkins
Branch: master

commit 203079caba3ba113009e6fcb0f8eea7247666761
Author: Matt Riedemann <email address hidden>
Date: Fri Nov 14 11:45:34 2014 -0800

    Pass expected_attrs to instance_get_active_by_window_joined

    This change sends the expected_attrs from the instance object
    API down into the DB API so the client can filter on which
    columns he actually needs to join when getting all instances
    in a given time window.

    The simple-tenant-usage API and _instance_usage_audit periodic
    task are the only things calling this API and they already
    provide specific expected_attrs so there is no risk of
    regression due to the change. The DB API is backwards
    compatible if columns_to_join is None, and expected_attrs
    defaults to None already.

    Change-Id: I3d8b5a1fd58783e465001bb5f10b5b87d69198d9
    Partial-Bug: #1383469

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/129732
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=466ddb115eb08e6507da8aeaf805f82c32111716
Submitter: Jenkins
Branch: master

commit 466ddb115eb08e6507da8aeaf805f82c32111716
Author: Matt Riedemann <email address hidden>
Date: Mon Oct 20 13:59:07 2014 -0700

    Limit InstanceList join to system_metadata in os-simple-tenant-usage

    The simple-tenant-usage API is calling get_active_by_window_joined with
    all of the default instance attribute fields when really only
    system_metadata is needed (to get the related flavor). That adds
    unnecessary load times on the DB query and data processing when
    constructing the instance list, especially given this is a query over a
    time range of instances, so as you add more instances to scale it's
    going to slow down.

    This simply limits the expected_attrs in the get_active_by_window_joined
    call to system_metadata so we don't load the other unnecessary
    attributes.

    Closes-Bug: #1383469

    Change-Id: Ic77d51cd45730ca664e68cfe0cb1671b5442d4a9