Comment 3 for bug 1689888

Ok - after doing some additional research I've confirmed the issue. I setup keystone locally and installed master (239bc3627cfb0546148e9d496f9e1536057052a7). I used a script to generate a bunch of data [0]. I ended up putting 10,000 project and 15,000 users in the system. Granted, I'm doing everything on my laptop (keystone + running with uwsgi, mysql, client, etc). My numbers won't be good representations of a real deployment, but I was able to notice a delta in the two APIs with respect to response time.

I create a script to time GET /v3/users and GET /v3/projects [1]. An additional script timed GET /v3/users/{user_id}, just based on some arbitrary user in the system [2].

I published the results without caching configured [3] and with caching configured [4].

Listing projects typically took 1 second while listing all users took between 3 and 4 seconds, on average. While there are slightly more users in the system, it does seem unproportional. I think additional investigation can be done to see where that is taking place. If it's somewhere in the identity controller or manager, we might be able to optimize python in that area (maybe we're doing unnecessary looping).

As far as the caching goes, you'll notice the listing of projects and users didn't differ when caching was enabled. It turns out we don't actually cache these sets. Keystone caches based on arguments, which is why you see the results of GET /v3/users/{user_id} improve after the first few calls (it goes from ~0.42 seconds to ~0.27 seconds).

While we might not be able to get caching implemented on those listing calls, I think we can certainly look at how we can optimize GET /v3/users/ and see if we can improve it's timing on large sets of data.

[0] https://gist.github.com/lbragstad/a4592f5fd52af1b0ad2b5f1fa57fb9ca#file-populate-py
[1] https://gist.github.com/lbragstad/a4592f5fd52af1b0ad2b5f1fa57fb9ca#file-time_list_projects_and_users-py
[2] https://gist.github.com/lbragstad/a4592f5fd52af1b0ad2b5f1fa57fb9ca#file-time_get_user-py
[3] https://gist.github.com/lbragstad/a4592f5fd52af1b0ad2b5f1fa57fb9ca#file-results-without-caching
[4] https://gist.github.com/lbragstad/a4592f5fd52af1b0ad2b5f1fa57fb9ca#file-results-with-caching