Yup, the problem is the name isn't passed to the server for filtering, you can recreate this with just a simple devstack setup and the default flavors. In this case I'm just doing:
nova --debug flavor-show m1.tiny
Here are the details:
1. Make a request to the server to get the flavor by the name, this fails with a 404 since the API expects the flavor id.
2. Now do a listing of the flavors, this is just passing is_public=None, it's not passing the name as a search filter. This returns all of the flavors.
3. The novaclient code had the list of flavors and then matched the one with the 'name' attribute and value being 'm1.tiny' so it then used that one to get the id (1 in this case) to make the flavor-get request again, this time with the id which then gets the response back we were looking for and it displays the flavor.
So the search filtering in novaclient is pretty broken when you're just searching by name. We only get lucky when there are <1000 records in the table we're searching on since we get the whole list back. The CONF.osapi_max_limit is what's pushing this over the top in this stress test scenario and why it starts failing after 1000.
Yup, the problem is the name isn't passed to the server for filtering, you can recreate this with just a simple devstack setup and the default flavors. In this case I'm just doing:
nova --debug flavor-show m1.tiny
Here are the details:
1. Make a request to the server to get the flavor by the name, this fails with a 404 since the API expects the flavor id.
DEBUG (session:248) REQ: curl -g -i -X GET http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ m1.tiny -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack- Nova-API- Version: 2.25" -H "X-Auth-Token: {SHA1}e2866846f 66ec299de60b3eb f17d931e7b9b957 9" :387) "GET /v2.1/a1c6a5d89 1d1420abec0227f 651b74c9/ flavors/ m1.tiny HTTP/1.1" 404 80 Request- Id: req-537e7042- 8626-485d- afb3-deb172b370 d0 Vary: X-OpenStack- Nova-API- Version Connection: keep-alive X-Openstack- Nova-Api- Version: 2.25 Date: Thu, 07 Apr 2016 01:10:24 GMT Content-Type: application/json; charset=UTF-8
DEBUG (connectionpool
DEBUG (session:277) RESP: [404] Content-Length: 80 X-Compute-
RESP BODY: {"itemNotFound": {"message": "Flavor m1.tiny could not be found.", "code": 404}}
2. Now do a listing of the flavors, this is just passing is_public=None, it's not passing the name as a search filter. This returns all of the flavors.
DEBUG (session:248) REQ: curl -g -i -X GET http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors? is_public= None -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack- Nova-API- Version: 2.25" -H "X-Auth-Token: {SHA1}e2866846f 66ec299de60b3eb f17d931e7b9b957 9" :387) "GET /v2.1/a1c6a5d89 1d1420abec0227f 651b74c9/ flavors? is_public= None HTTP/1.1" 200 1720 Request- Id: req-5ea197c2- 1441-4e42- 8c9f-3d2b70fa1a be Vary: X-OpenStack- Nova-API- Version Connection: keep-alive X-Openstack- Nova-Api- Version: 2.25 Date: Thu, 07 Apr 2016 01:10:25 GMT Content-Type: application/json 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 1", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 1", "rel": "bookmark"}], "name": "m1.tiny"}, {"id": "2", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 2", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 2", "rel": "bookmark"}], "name": "m1.small"}, {"id": "3", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 3", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 3", "rel": "bookmark"}], "name": "m1.medium"}, {"id": "4", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 4", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 4", "rel": "bookmark"}], "name": "m1.large"}, {"id": "42", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 42", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 42", "rel": "bookmark"}], "name": "m1.nano"}, {"id": "5", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 5", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 5", "rel": "bookmark"}], "name": "m1.xlarge"}, {"id": "84", "links": [{"href": "http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 84", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 84", "rel": "bookmark"}], "name": "m1.micro"}]}
DEBUG (connectionpool
DEBUG (session:277) RESP: [200] Content-Length: 1720 X-Compute-
RESP BODY: {"flavors": [{"id": "1", "links": [{"href": "http://
3. The novaclient code had the list of flavors and then matched the one with the 'name' attribute and value being 'm1.tiny' so it then used that one to get the id (1 in this case) to make the flavor-get request again, this time with the id which then gets the response back we were looking for and it displays the flavor.
DEBUG (session:248) REQ: curl -g -i -X GET http:// 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 1 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack- Nova-API- Version: 2.25" -H "X-Auth-Token: {SHA1}e2866846f 66ec299de60b3eb f17d931e7b9b957 9" :387) "GET /v2.1/a1c6a5d89 1d1420abec0227f 651b74c9/ flavors/ 1 HTTP/1.1" 200 422 Request- Id: req-446b0bbc- 1f95-471f- 821a-3c1fdc141b 3b Vary: X-OpenStack- Nova-API- Version Connection: keep-alive X-Openstack- Nova-Api- Version: 2.25 Date: Thu, 07 Apr 2016 01:10:25 GMT Content-Type: application/json 9.5.124. 163:8774/ v2.1/a1c6a5d891 d1420abec0227f6 51b74c9/ flavors/ 1", "rel": "self"}, {"href": "http:// 9.5.124. 163:8774/ a1c6a5d891d1420 abec0227f651b74 c9/flavors/ 1", "rel": "bookmark"}], "ram": 512, "OS-FLV- DISABLED: disabled" : false, "vcpus": 1, "swap": "", "os-flavor- access: is_public" : true, "rxtx_factor": 1.0, "OS-FLV- EXT-DATA: ephemeral" : 0, "disk": 1, "id": "1"}}
DEBUG (connectionpool
DEBUG (session:277) RESP: [200] Content-Length: 422 X-Compute-
RESP BODY: {"flavor": {"name": "m1.tiny", "links": [{"href": "http://
---
So the search filtering in novaclient is pretty broken when you're just searching by name. We only get lucky when there are <1000 records in the table we're searching on since we get the whole list back. The CONF.osapi_ max_limit is what's pushing this over the top in this stress test scenario and why it starts failing after 1000.