Fix for 1694897 broke existing agent function

[Sridhar Venkat]

Due to change made for 1694897 - (https://github.com/openstack/neutron/commit/b50fed17fa338f4137646c9c8d8e47634b7f5ff7#diff-47274e5d515466fd8a373b382009b419) broke existing functionality.

From neutron server log :
2017-07-26 08:17:35.737 4946 INFO neutron.wsgi [req-13b37d24-f287-44e1-964b-89a740148970 09190e437c2fd33046047c5458396ba91f2468cc22a1c33bbc12a0148d0885e4 5f0805bffd4349c4a53a81a5516c5631 - - -] 127.0.0.1 - - [26/Jul/2017 08:17:35] "GET /v2.0/networks.json HTTP/1.1" 200 936 19.272352
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource [req-8669ee8b-eb35-4178-8a30-bd99bdd9efa7 09190e437c2fd33046047c5458396ba91f2468cc22a1c33bbc12a0148d0885e4 5f0805bffd4349c4a53a81a5516c5631 - - -] index failed: No details.
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource Traceback (most recent call last):
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource File "/usr/lib/python2.7/site-packages/neutron/api/v2/resource.py", line 91, in resource
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource args['id'] = '.'.join([args['id'], fmt])
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource KeyError: 'id'
2017-07-26 08:17:46.385 4949 ERROR neutron.api.v2.resource

args variable is empty dictionary : {}

It is due to this code in the same resource.py resource method:
        route_args = request.environ.get('wsgiorg.routing_args')
        if route_args:
            args = route_args[1].copy()
        else:
            args = {}

args can be empty dictionary. args['id'] will be invalid.

This problem is found in ocata release of OpenStack.

[Sridhar Venkat (svenkat)]

Did more analysis using pdb. In failing environment seeing this :
> /usr/lib/python2.7/site-packages/neutron/api/v2/resource.py(93)resource()
-> if fmt is not None and fmt not in format_types:
(Pdb) l
 88 # to tackle it without breaking API backwards compatibility.
 89 import pdb; pdb.set_trace()
 90 print args
 91 print fmt
 92 print format_types
 93 -> if fmt is not None and fmt not in format_types:
 94 args['id'] = '.'.join([args['id'], fmt])
 95
 96 method = getattr(controller, action)
 97 result = method(request=request, **args)
 98 except Exception as e:
(Pdb) n
> /usr/lib/python2.7/site-packages/neutron/api/v2/resource.py(94)resource()
-> args['id'] = '.'.join([args['id'], fmt])
(Pdb) p fmt
u'json.json'
(Pdb) p format_types
{'json': 'application/json'}

Since json.json is not in format_types, args['id'] is expected and it fails.

[Sridhar Venkat (svenkat)]

Above comment is from ocata environment. In Pike :
> /usr/lib/python2.7/site-packages/neutron/api/v2/resource.py(93)resource()
-> if fmt is not None and fmt not in format_types:
(Pdb) l
 88 # to tackle it without breaking API backwards compatibility.
 89 import pdb; pdb.set_trace()
 90 print args
 91 print fmt
 92 print format_types
 93 -> if fmt is not None and fmt not in format_types:
 94 args['id'] = '.'.join([args['id'], fmt])
 95
 96 method = getattr(controller, action)
 97 result = method(request=request, **args)
 98 except Exception as e:
(Pdb) n
> /usr/lib/python2.7/site-packages/neutron/api/v2/resource.py(96)resource()
-> method = getattr(controller, action)
(Pdb) p fmt
u'json'
(Pdb) p format_types
{'json': 'application/json'}

if condition is not satisfied, so do not see this problem in Pike.

[Sridhar Venkat (svenkat)]

In failing case: variable route_args : (<routes.util.URLGenerator object at 0x10034269a50>, {'action': u'index', 'controller': <wsgify at 1100392530832 wrapping <function resource at 0x100347fe668>>, 'format': u'json.json'})

[Brian Haley (brian-haley)]

Just adding links to the original bug and upstream review:

https://bugs.launchpad.net/zun/+bug/1694897
https://review.openstack.org/#/c/470016/

[Ihar Hrachyshka (ihar-hrachyshka)]

Can you give us full info about the request in question? I wonder how you get json.json in wsgi context dictionary.

[Sridhar Venkat (svenkat)]

dump of request :
GET /v2.0/agents.json.json HTTP/1.0
Accept: application/json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Type: text/plain
Host: 127.0.0.1:9696
User-Agent: python-neutronclient
X-Auth-Token: gAAAAABZeHalBtTzcEv94enqwcUcD-jAmBfU_A8rBr3STNUdDYqJ6t3CX-vIBHWWFhiBx4z8elau_9ZKTXeUBC3GxTNUFRMump0AaM_D-1i33jpK0EqaFb3lM0MVlwFS-6usbg7nNhmvfw3rT7Xs0awUDihdKf5k0r7M0OHVmqCxNMQ5DMLcDiba6hEDOSExLLGydX8iOw0qxnpLOOCDQZhw1EbdmWPwJ_ONFql-dqhMufuLpLTNtaM
X-Domain-Id: None
X-Domain-Name: None
X-Identity-Status: Confirmed
X-Is-Admin-Project: False
X-Project-Domain-Id: 03bd7eef7bd943c29a273b82d7ef12c4
X-Project-Domain-Name: Service
X-Project-Id: 5f0805bffd4349c4a53a81a5516c5631
X-Project-Name: service
X-Role: admin
X-Roles: admin
X-Service-Catalog: <removed>
X-Tenant: service
X-Tenant-Id: 5f0805bffd4349c4a53a81a5516c5631
X-Tenant-Name: service
X-User: neutron
X-User-Domain-Id: 03bd7eef7bd943c29a273b82d7ef12c4
X-User-Domain-Name: Service
X-User-Id: 09190e437c2fd33046047c5458396ba91f2468cc22a1c33bbc12a0148d0885e4
X-User-Name: neutron
(<routes.util.URLGenerator object at 0x100437447d0>, {'action': u'index', 'controller': <wsgify at 1100652491600 wrapping <function resource at 0x10043ffb668>>, 'format': u'json.json'})
{}
json.json
{'json': 'application/json'}
> /usr/lib/python2.7/site-packages/neutron/api/v2/resource.py(96)resource()
-> args['id'] = '.'.join([args['id'], fmt])

(Pdb) p fmt
u'json.json'
(Pdb) /

[Sridhar Venkat (svenkat)]

neutron client used is :
/usr/lib/python2.7/site-packages/python_neutronclient-6.3.0-py2.7.egg-info

[Sridhar Venkat (svenkat)]

Correction to previous comment.. it is 6.1.0.

neutron --version shows version as 6.1.0.

[Sridhar Venkat (svenkat)]

Any updates on this defect?

[Sridhar Venkat (svenkat)]

Any updates?

[Matthew Edmonds (edmondsw)]

Why is the request GET /v2.0/agents.json.json and not GET /v2.0/agents.json ?

[Matthew Edmonds (edmondsw)]

looks like that was a neutronclient bug fixed by https://github.com/openstack/python-neutronclient/commit/a905f755b995f358116a11f77de71893b0601937, but ocata is capped at an older version of neutronclient than was fixed. Do we need to backport the fix in neutronclient, then?

[Bernard Cafarelli (bcafarel)]

Marking as fixed, Ocata has been maintenance-only for some time now (and Pike too) and newer releases are not affected