openstack_dashboard.usage.quotas.tenant_quota_usages fetches to many quotas and degrades performances

[Yves-Gwenael Bourhis]

When looking at the keypair dashboard, or router dashboard, etc... openstack_dashboard.usage.quotas.tenant_quota_usages queries also cinder, etc... and therefore makes tons of unused and unnecessary api calls.

It slows down panels very heavily.

I did a quick and dirty test on some dashboards (with a local horizon targeted on a production environment's APIs) by modifying the tenant_quota_usages function to retrieve only the desired quotas and usages for the page I need, and it turns out that rendering a page is between 3 to five time faster. Just enormous.

We really do not need to fetch all usages and quotas when retrieving keypairs, etc...

This function should accept extra parameters to get only the desired quotas and usages.

However this is a huge task because it requires modifying ALL dashboards.

OPENSTACK_HYPERVISOR_FEATURES['enable_quotas'] and OPENSTACK_NEUTRON_NETWORK['enable_quotas'] set to False is not a solution because it prevents fetching these quotas whatever the dashboard.

The solution is e.g. not fetching network quotas and usages on the volumes dashboard or cinder quotas and usages on the keypairs dashboard, etc...

Quick tests show that the performance impact is simply more than tremendously gigantic. e.g.: My "routers" page with the current function takes 20 to 30 seconds to render (just catastrophic), and only 3 to 5 seconds If I get only the the desired quotas and usages required only for this dashboard.

[Yves-Gwenael Bourhis (yves-gwenael-bourhis)]

This current bug should be taken into account for this blueprint : https://blueprints.launchpad.net/horizon/+spec/make-quotas-great-again

tenant_quota_usages should not fetch ALL quotas and usages but only the ones needed for the dashboard calling it.

[Yves-Gwenael Bourhis (yves-gwenael-bourhis)]

The "allowed" methods on LinkAction buttons (like CreateNetwork.allowed) call "usages = quotas.tenant_quota_usages(request)".

These "allowed" methods should request usages only for what they are concerned with (volume usages and all others are not needed for CreateNetwork.allowed. CreateNetwork is an example, but ALL LinkAction or Actions are concerned everywhere where they fetch tenant_quota_usages.

These methods should not do a "if usages.get('networks', {})" etc... they should pass the parameter of usages they want and tenant_quota_usages should restrict to the strict and necessary API calls only.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/456415

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/456416

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/456415
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=81891697c6c8d2cf1ee0bfceb3512e174624bc00
Submitter: Jenkins
Branch: master

commit 81891697c6c8d2cf1ee0bfceb3512e174624bc00
Author: Akihiro Motoki <email address hidden>
Date: Wed Apr 12 16:34:06 2017 +0000

    Retrieve quota and usage only for quota-enabled resources

    Previously _get_tenant_(compute|network|volume)_usage are callled
    regardless of resources are listed in enabled_quotas.
    This commit retrives usage information only for resources listed
    in eanbled_quotas.

    floating_ips and security_groups were returned even when
    they are included in disabled_quotas. They are not needed
    and they are cleanup in this commit.

    All (*_)QUOTA_FILEDS were previously a list but it is always used
    after converting into a set, so this commit changes (*_)QUOTA_FIELDS
    into a set in the initial definitions.

    This is a preparation for fix of bug 1675504

    Partial-Bug: #1675504
    Change-Id: I41cdabde477d87aa8b35d1c908e18a69454286c3

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/456416
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=359467b4013bb4f89a6a1309e6eda89459288986
Submitter: Jenkins
Branch: master

commit 359467b4013bb4f89a6a1309e6eda89459288986
Author: Akihiro Motoki <email address hidden>
Date: Wed Apr 12 18:10:20 2017 +0000

    Retrieve quota and usage only for resources really required

    tenant_quota_usage() is used to retrieve quota and usage
    to determine if a resource can be created.
    However, tenant_quota_usage retrieves quota and usage for
    all resources and it can be a performance problem.

    This commit allows to load quota and usage only for resources
    which are actually required.

    Closes-Bug: #1675504
    Change-Id: Iab7322a337a451a1a040cc2f4b55cc319b1ffc4c

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/horizon 12.0.0.0b3 development milestone.