Comment 7 for bug 1182481

In my scenario, token flush is not making much improvement. The queries are taking few miliseconds to finish:

2013-05-30 16:39:34 PDT,12998,[unknown]LOG: statement: SELECT token.id AS token_id, token.expires AS token_expires, token.extra AS token_extra, token.valid AS token_valid, token.user_id AS token_user_id, token.trust_id AS token_trust_id
        FROM token
        WHERE token.expires > '2013-05-30T23:39:34.813557'::timestamp AND token.valid = false
2013-05-30 16:39:34 PDT,12998,[unknown]LOG: duration: 19.337 ms

However, what am noticing here is if I de-provision all the launched instances, the navigation through various tabs finishes in miliseconds. If I keep launching VMs and at this point am on 100, the response time goes between 40-60 seconds. Same behavior is seen with nova list command. So, it looks a factor of instances or objects in the openstack.

nova list
2013-05-30 10:07:46.063 37206 INFO nova.osapi_compute.wsgi.server [-] (37206) accepted ('10.1.56.12', 40240)
2013-05-30 10:08:26.249 INFO nova.osapi_compute.wsgi.server [req-fa0cf5a9-270a-4a03-92fe-e0d124919e16 2efad4b253f64b4dae65a28f45438d93 4f342d62fff843fab63dc03316d34251] 10.1.56.12 "GET /v2/4f342d62fff843fab63dc03316d34251/servers/detail HTTP/1.1" status: 200 len: 153050 time: 40.1845958

Observation on nova list:
1. Overall response time to enumerate 100 launched instance takes 40 seconds
2. Throughout this 40 seconds duration, keystone-all is taking 70% cpu (in a 32 core system)
3. I took a count of keystone.token rows before and after I executed the command ‘nova list’. The counted bumped up from 46,671 to 46,884 (around 200 rows). Does it mean it is going to keystone for every instance enumeration.