create token API is not doing proper input validation

[Matthew Edmonds]

HTTP 500 being returned when the request body for POST /v3/auth/tokens has an empty string in place of one of the dicts that should be passed in. This shows that the code is not doing proper input validation. It should detect the user error and return an HTTP 400. Here's an example where project domain is "" instead of {"id": "default"}:

# curl -1 -k -i -X POST https://localhost:5000/v3/auth/tokens -H "Accept: application/json" -H "Content-Type: application/json" -d '{"auth": {"scope": {"project": {"name": "myproj", "domain": ""}}, "identity": {"methods": ["password"], "password": {"user": {"domain": {"name": "Default"}, "name": "myuser", "password": "mypassword"}}}}}'
HTTP/1.1 500 Internal Server Error
Date: Tue, 10 May 2016 20:39:53 GMT
Server: Apache
Vary: X-Auth-Token
x-openstack-request-id: req-a4961a66-b545-407e-9aa3-7575e38c252c
Content-Length: 143
Connection: close
Content-Type: application/json

{"error": {"message": "An unexpected error prevented the server from fulfilling your request.", "code": 500, "title": "Internal Server Error"}}

Logs show:

2016-05-10 16:39:53.716 2951 INFO keystone.common.wsgi [req-a4961a66-b545-407e-9aa3-7575e38c252c - - - - -] POST https://localhost:5000/v3/auth/tokens
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi [req-a4961a66-b545-407e-9aa3-7575e38c252c - - - - -] 'unicode' object has no attribute 'get'
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi Traceback (most recent call last):
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 249, in __call__
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi result = method(context, **params)
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 392, in authenticate_for_token
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi auth_info = AuthInfo.create(context, auth=auth)
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 137, in create
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi auth_info._validate_and_normalize_auth_data(scope_only)
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 305, in _validate_and_normalize_auth_data
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi self._validate_and_normalize_scope_data()
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 247, in _validate_and_normalize_scope_data
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi project_ref = self._lookup_project(self.auth['scope']['project'])
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 210, in _lookup_project
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi domain_ref = self._lookup_domain(project_info['domain'])
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi File "/usr/lib/python2.7/site-packages/keystone/auth/controllers.py", line 172, in _lookup_domain
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi domain_id = domain_info.get('id')
2016-05-10 16:39:53.717 2951 ERROR keystone.common.wsgi AttributeError: 'unicode' object has no attribute 'get'

Note: you can also get HTTP 500 if you replace other dicts in the request, e.g. {"user": ""}

[Steve Martinelli (stevemar)]

We could use a jsonschema here, but it won't be easy - there are quite a few combinations to over.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/315894

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/315894
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=fe3b4c0f653318b7262423386c169a5d650ce08d
Submitter: Jenkins
Branch: master

commit fe3b4c0f653318b7262423386c169a5d650ce08d
Author: Ryosuke Mizuno <email address hidden>
Date: Fri May 13 14:46:45 2016 +0900

    Add the validation rules when create token

    HTTP 500 being returned when the request body attibute for
    POST /v3/auth/tokens has an empty string instead of dict.
    This patch fix when the domain and the project in the scope has
    an empty string.

    Change-Id: I11031a5a19acb9e336721d69b59e7e6f691eb2a4
    Partial-Bug: #1580338

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/325086

[Lance Bragstad (lbragstad)]

Automatically unassigning due to inactivity. Looks like the patch in review needs some attention. Unassigning so that others can pick this up if they have the bandwidth.

[Steve Martinelli (stevemar)]

Brant's new patch: https://review.openstack.org/#/c/404806/

[Lance Bragstad (lbragstad)]

FWIW - we do use jsonschema for validating token responses [0]. The schema for scoped tokens is kind of intense but it is possible. I wanted to mention it here in case it helped with using jsonschema to properly validate token requests.

[0] https://github.com/openstack/keystone/blob/fe830ffe61912bd6e3d06c1b1eb82ca23f9314eb/keystone/tests/unit/test_v3.py#L127

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/404806
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=d16ec35378494f06a983a4c7f8566719bcf6ac3e
Submitter: Jenkins
Branch: master

commit d16ec35378494f06a983a4c7f8566719bcf6ac3e
Author: Brant Knudson <email address hidden>
Date: Wed Nov 30 08:47:01 2016 -0600

    Validate token issue input

    The server wasn't validating the input to `POST /v3/auth/tokens`
    and instead allowing invalid input to be sent into the code which
    only haphazardly checked the types and values of the data, leading
    to confusing errors such as 500 Internal Server Error rather than
    400 Bad Request as expected, and also potentially security issues.

    Closes-Bug: 1580338
    Change-Id: I34146659ebb912485716a7d07f7ae7800a5cb9ac

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 11.0.0.0b2 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by David Stanek (<email address hidden>) on branch: master
Review: https://review.openstack.org/325086
Reason: The tagged bug was fixed with the review Steve mentioned.