Limit listed Swift objects in Horizon

[Alvaro Uria]

There's is a setting in Horizon (API_RESULT_LIMIT) to limit the number of items returned by Swift when listing containers and objects. That value is hard-coded to 1000 and it would be nice to be able to tweak it via a Juju config change.

OTOH, it looks like there is a bug in how Horizon itself requests a limited list of items:

* swift_get_containers: retrieves all the accounts and containers, and returns the limited number of containers (max: 1000)
https://github.com/openstack/horizon/blob/ddb5913c1edaefb9f16d6305c5c30fc454e1de99/openstack_dashboard/api/swift.py#L139

* swift_get_objects: retrieves all the containers and objects, and returns the limited number of objects (max: 1000)
https://github.com/openstack/horizon/blob/ddb5913c1edaefb9f16d6305c5c30fc454e1de99/openstack_dashboard/api/swift.py#L217
https://github.com/openstack/horizon/blob/ddb5913c1edaefb9f16d6305c5c30fc454e1de99/openstack_dashboard/api/swift.py#L77
https://github.com/openstack/swift/blob/9263e7daf5d663e56052abde89ab543a12898f3f/swift/common/internal_client.py#L892
https://github.com/openstack/swift/blob/9263e7daf5d663e56052abde89ab543a12898f3f/swift/common/internal_client.py#L865
https://github.com/openstack/swift/blob/9263e7daf5d663e56052abde89ab543a12898f3f/swift/common/internal_client.py#L783

Note the last link shows "base_request" and its arguments. "limit" is not one of them.

[Alvaro Uria (aluria)]

The expected behavior is that API_RESULT_LIMIT limits the number of accounts/containers/objects retrieved from Swift.

[Drew Freiberger (afreiberger)]

 I'm not certain I agree with the assessment on the swift calls based on the evidence provided. It appears that the get_container and get_account are being called with limit=$limit+1 as a kwarg which should have the swift-proxy api limiting the response to limit+1.

Line 141 and 147 of the os_dash/api/swift.py file linked in the first URL send limit=limit+1 as a kwarg to the swift GET call, and then chop off the extra 1 entry which then returns True (responses limited by value).

I'm thinking this will work as intended, if you reduce the API_RESULT_LIMIT, it should reduce the results returned to the horizon unit.

[Drew Freiberger (afreiberger)]

Though, +1 on adding API_RESULT_LIMIT to be modifiable in the charm config.

From local_settings.py (copied from juno):

# The number of Swift containers and objects to display on a single page before
# providing a paging element (a "more" link) to paginate results.
API_RESULT_LIMIT = 1000

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/579659

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579659
Committed: https://git.openstack.org/cgit/openstack/charm-openstack-dashboard/commit/?id=e58a1b2a3dd40c6f65c9d4581ef0cf0ed104e11f
Submitter: Zuul
Branch: master

commit e58a1b2a3dd40c6f65c9d4581ef0cf0ed104e11f
Author: Shane Peters <email address hidden>
Date: Mon Jul 2 13:38:11 2018 -0400

    Add ability to configure api_result_limit

    In clouds with many containers or objects, listing them via Horizon
    can cause a signficant increase of system load.

    This patch enables configuration of the API_RESULT_LIMIT setting within
    Horizon. This limits the maximum number of objects to display on a
    single page before providing a paging element to paginate results.

    Change-Id: Ifaf39d6c9bf549428afd7653243c82cd719956f6
    Closes-Bug: 1775002