Identity API v3 filtering support

[Cindy Lu]

Identity API v3 provides filtering features that Horizon does not leverage yet.

(side note: v3 does not seem to support pagination for projects though v2 does... https://blueprints.launchpad.net/horizon/+spec/v3-pagination-support.)

Filtering API:
https://github.com/openstack/identity-api/blob/master/v3/src/markdown/identity-api-v3.md

They can be implemented for users and projects dashboards in a similar fashion as the Instances filter by column.

Curl example:
curl -H "X-Auth-Token:<<TOKEN>>" http://localhost:5000/v3/projects?name=admin
/v3/users
/v3/projects

* There is no agreed upon standard for how to handle filtering across the board => inconsistent. For example, Nova Instance API allows you to filter on substring but Glance Image API only allow you to filter on exact string match.

[Cindy Lu (clu-m)]

Related to: https://blueprints.launchpad.net/horizon/+spec/table-filtering-by-api-query

[Lin Hua Cheng (lin-hua-cheng)]

I may be wrong but I don't think keystone implemented filtering and pagination in V3, although it is documented in the spec.

[Cindy Lu (clu-m)]

no pagination yet, but the filtering works in v3. I tried it locally: "http://localhost:5000/v3/projects?name=admin"

[Justin Pomeroy (jpomero)]

I think we can get by without pagination if we can come up with a default filter so that all users are only listed if the user explicitly requests that. The problem I'm seeing is that you go to the users page and initially there is no filter so every single user is going to be displayed. When there are a LOT of users this is a performance issue. Same goes for any of the non-paged tables really. I'm glad to see v3 supports a "contains" type filter which is probably what we'd want to use. Looking at the filter possibilities though I'm not really coming up with an obvious default filter.

[Justin Pomeroy (jpomero)]

Another option, instead of trying to guess what a good default filter is, would be to just cut off the list at some reasonable max value and then show a warning telling the user that the list is truncated and to narrow down the results using a filter.

"500 of the 2,345 total users are displayed. Use a filter to narrow down the results."

[Justin Pomeroy (jpomero)]

I have a fix that adds API filtering to the Users and Projects tables, but it depends on the initial API filtering support: https://review.openstack.org/#/c/81574/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/112744

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Lyle (<email address hidden>) on branch: master
Review: https://review.openstack.org/112744
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Timur Sufiev (tsufiev-x)]

I'd like to share our plans for fixing Identity tables in Mitaka.

Keystone will not provide pagination to Horizon and Horizon will not implement pagination on Keystone results.
Keystone configuration file will expose hard limits for the number of users, projects, roles, etc. returned to Horizon. These limits can be configured independently of one another (e.g. hard limit of 50 users, 100 projects, 10 roles). Default configuration limits should be provided and the operator should be made aware that higher hard limits will result in additional latency on displaying this data in Horizon. Hard limits for returned resources is already implemented in Keystone.

Keystone must support server-side filtering, because the client-side filtering that is available will only filter/sort inside of each page. This does not provide a solution for the use case when there are a large number of Keystone users (e.g. 10,000) and the hard limit of users is returned (e.g. 200) and then filtered. For example, I am looking for user adamyoung. Keystone returns an arbitrary 200 users, in accordance with the hard limit supplied. I then filter the returned users to only show users which contain the string “adam”. Because the filtering is performed on the client side, it is unlikely that the user I am searching for will be returned.

Horizon will, by default, request and display no more the hard limit of users/projects/roles from Keystone as provided in the configuration file – regardless of whether a filter is provided. Once the filter has been supplied by the user, Horizon will query Keystone using the filter details to return a new list of resources. This satisfies the use case where the user provides a filter (again, e.g. “adam”) and the filtering is performed server-side (Keystone only returns users containing the string “adam), but there are more than the hard limit number worth of returned entries (e.g. Keystone is configured to return 200 entries, but 280 are available). In this case, it is necessary to notify Horizon that there are additional users which have not been displayed so that the user querying for “adam” knows the user they are seeking may not be visible in the results, but does exist.

The series of patches enabling/fixing hard limits of Keystone entities is here: https://review.openstack.org/#/c/234849/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/252488

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Lyle (<email address hidden>) on branch: master
Review: https://review.openstack.org/252488
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Lars Erik Pedersen (pedersen-larserik)]

Are there any updates on this?

[Paul Karikh (pkarikh)]

@pedersen-larserik I'm wating for this fix in python-keystoneclient: https://review.openstack.org/#/c/293048/

Looks like keystone community is still discussing best approach for such fixes, because there is another one fix in python-keystoneclient for other bug. Main disscussion thread is here: https://review.openstack.org/#/c/261188/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Rob Cresswell (<email address hidden>) on branch: master
Review: https://review.openstack.org/252488
Reason: Due to inactivity and merge conflict

[Ivan Kolodyazhny (e0ne)]

We need to verify if it's still actual.

[David Lyle (david-lyle)]

Wasn't this covered by https://blueprints.launchpad.net/horizon/+spec/server-side-filtering? looking on master I see server side filtering for all panels except domains.