Do not display all if resource exceeds the osapi_max_limit.

Bug #1726770 reported by TommyLike on 2017-10-24
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-cinderclient
Wishlist
TommyLike

Bug Description

We added the feature 'List all the request items when the list is over osapi_max_limit'[1] since Kilo.

It's a nice addition to the list command which enables the administrator to show all the resources he has, but there comes another question that the administrator could have thousands of volumes in the real environment and trigger such an operation ( request repeatedly and assembly resources in the client ) could consume a lot of resources (see the picture added below), and it's also not easy to handle the output if over 1000 thousands records are displayed.

It's also an inconsistent as lots of the openstack projects only display records <= osapi_max_limit once a time.

[1] https://bugs.launchpad.net/cinder/+bug/1342192
[2] nova: https://github.com/openstack/python-novaclient/blob/master/novaclient/base.py#L250
[3] manila: https://github.com/openstack/python-manilaclient/blob/master/manilaclient/base.py#L56

TommyLike (hu-husheng) wrote :
Changed in cinder:
assignee: nobody → TommyLike (hu-husheng)
TommyLike (hu-husheng) on 2017-10-24
summary: - cinder list all command consumes lots of resource when handling lots of
- volumes
+ Do not display all resource if response exceeds the max size.
summary: - Do not display all resource if response exceeds the max size.
+ Do not display all if resource exceeds the osapi_max_limit.
Matt Riedemann (mriedem) wrote :

https://review.openstack.org/#/c/134133/ is arguably implementing the wrong default behavior.

We have something similar in python-novaclient where if you pass limit=-1 then that means "no limit" and the client will page through the results automatically and display everything until there are no more pages.

I'd suggest changing the default limit behavior in python-cinderclient to be similar, so that the user opts into getting unlimited results, else the default is just enforced via the configurable limit on the server side. It would be a behavior change though so a release note would definitely be needed, and likely at least a minor version change in the client.

no longer affects: cinder
TommyLike (hu-husheng) on 2017-10-24
Changed in python-cinderclient:
assignee: nobody → TommyLike (hu-husheng)
Sean McGinnis (sean-mcginnis) wrote :

It seems kind of odd to me if the user asks for an unfiltered list that we would not return all the items.

But that said, I suppose I could see this being a usability scaling issue for very large deployments. I guess as long as we make it clear that there is more data not returned (and how to get the rest) then I'd be fine with changing it.

Changed in python-cinderclient:
importance: Undecided → Wishlist
Eric Harney (eharney) wrote :

Yeah, I strongly disagree with the idea that we should limit the list shown in the client just because other openstack projects behave that way. That sounds like a bug, which we addressed.

If there is an issue with the client using lots of resources for this, we can surely improve that.

But let's stick with letting "cinder list" do what it sounds like it does rather than artificially limiting it.

Fix proposed to branch: master
Review: https://review.openstack.org/514896

Changed in python-cinderclient:
status: New → In Progress
Eric Harney (eharney) wrote :

-> Incomplete, pending more info on perf issues

Changed in python-cinderclient:
status: In Progress → Incomplete

Change abandoned by Sean McGinnis (<email address hidden>) on branch: master
Review: https://review.openstack.org/514896
Reason: This review is > 4 weeks without comment and currently blocked by a core reviewer with a -2. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and contacting the reviewer with the -2 on this review to ensure you address their concerns.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers