Invalid query parameter could lead to HTTP 500

Bug #1746202 reported by Zhenyu Zheng
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Undecided
TommyLike
OpenStack Compute (nova)
Fix Released
Undecided
Zhenyu Zheng
OpenStack Shared File Systems Service (Manila)
Fix Released
Undecided
zhongjun

Bug Description

Invalid query parameter could lead to HTTP 500, although Nova used JSON Schema verification
to check input query params, but query like:
GET /servers?limit=%88 will still lead to HTTP 500, as it failed to parse at webob which is
pre JSON Schema check.

GET http://10.76.150.18/compute/v2.1/servers/detail?limit=%88

Response:

{
    "computeFault": {
        "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<type 'exceptions.UnicodeDecodeError'>",
        "code": 500
    }
}

Traceback:

DEBUG nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Unexpected exception in API method: UnicodeDecodeError: 'utf8' codec can't decode byte 0x89 in position 0: invalid start byt
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 803, in wrapped
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 171, in wrapper
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi req.GET.dict_of_lists(),
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python2.7/dist-packages/webob/request.py", line 833, in GET
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi vars = GetDict(data, env)
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python2.7/dist-packages/webob/multidict.py", line 287, in __init__
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi MultiDict.__init__(self, data)
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python2.7/dist-packages/webob/multidict.py", line 38, in __init__
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi items = list(args[0])
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python2.7/dist-packages/webob/compat.py", line 126, in parse_qsl_text
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi yield (x.decode(encoding), y.decode(encoding))
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi return codecs.utf_8_decode(input, errors, True)
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi UnicodeDecodeError: 'utf8' codec can't decode byte 0x89 in position 0: invalid start byte
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: ERROR nova.api.openstack.wsgi
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: INFO nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API lo
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: <type 'exceptions.UnicodeDecodeError'>
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: DEBUG nova.api.openstack.wsgi [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API l
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: <type 'exceptions.UnicodeDecodeError'> {{(pid=4377) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1079}}
Jan 30 17:46:56 kevin-dev <email address hidden>[4374]: INFO nova.api.openstack.requestlog [None req-ee355759-13c3-4f63-a41f-920d7385878d admin admin] 10.8.4.18 "GET /compute/v2.1/servers/detail?limit=%89" status: 500 len: 202 microversion: 2.49 time: 0.531050

Tags: api
Changed in nova:
assignee: nobody → Zhenyu Zheng (zhengzhenyu)
Changed in nova:
status: New → In Progress
affects: nova → cinder
affects: cinder → nova
TommyLike (hu-husheng)
Changed in cinder:
assignee: nobody → TommyLike (hu-husheng)
Revision history for this message
TommyLike (hu-husheng) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/539387

Changed in cinder:
status: New → In Progress
zhongjun (jun-zhongjun)
Changed in manila:
assignee: nobody → zhongjun (jun-zhongjun)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/539406

Changed in manila:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/539408

Changed in nova:
assignee: Zhenyu Zheng (zhengzhenyu) → Yikun Jiang (yikunkero)
Changed in nova:
assignee: Yikun Jiang (yikunkero) → Zhenyu Zheng (zhengzhenyu)
Changed in manila:
milestone: none → queens-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.openstack.org/539406
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=bf1998898453f208882a95bf88cd157274e7dbf1
Submitter: Zuul
Branch: master

commit bf1998898453f208882a95bf88cd157274e7dbf1
Author: zhongjun <email address hidden>
Date: Wed Jan 31 11:02:44 2018 +0800

    Fix UnicodeDecodeError when decode API input

    Convert UnicodeDecodeError to HTTPBadRequest in
    FaultWrapper.

    Change-Id: I826f05084b0a0ef170ef293d382868409b96ed3d
    Closes-Bug: #1746202

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/539387
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=0e2193982ab1e4941ed82c3785e306eb8e4e46ec
Submitter: Zuul
Branch: master

commit 0e2193982ab1e4941ed82c3785e306eb8e4e46ec
Author: TommyLike <email address hidden>
Date: Wed Jan 31 09:41:13 2018 +0800

    Fix UnicodeDecodeError when decode API input

    Convert UnicodeDecodeError to HTTPBadRequest in
    FaultWrapper.

    **NOTE**: Cinder will raise 500 error when failed to decode invalid
    unicode character at anyplace where try to collect url
    parematers as below:

    ```
    params = req.params
    ```

    This patch converts this kind of exception into BadRequest, plus
    an explicit error message.

    Change-Id: I816f05084b0a0ef670ef293d381868409b96ed7d
    Closes-Bug: #1746202

Changed in cinder:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/cinder 12.0.0.0rc1

This issue was fixed in the openstack/cinder 12.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 6.0.0.0rc1

This issue was fixed in the openstack/manila 6.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/539164
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6029ccd44e500bd1d3ba41821793216cb4eac6dd
Submitter: Zuul
Branch: master

commit 6029ccd44e500bd1d3ba41821793216cb4eac6dd
Author: Kevin_Zheng <email address hidden>
Date: Tue Jan 30 17:28:01 2018 +0800

    Invalid query parameter could lead to HTTP 500

    Invalid query parameter could lead to HTTP 500, although
    Nova used JSON Schema verification to check input query
    params, but query like: GET /servers?limit=%88 will still
    lead to HTTP 500, as it failed to parse at webob which is
    pre JSON Schema check.

    Partial-Bug: #1746202

    Change-Id: I11b94a1aaeb67dc1a5abdcf0af5961ee8942a50a

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Reviewed: https://review.openstack.org/539408
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ea006b13a7ad5229a3dcdbba0e7aef438e873b6c
Submitter: Zuul
Branch: master

commit ea006b13a7ad5229a3dcdbba0e7aef438e873b6c
Author: Yikun Jiang <email address hidden>
Date: Wed Jan 31 10:50:46 2018 +0800

    [Placement] Invalid query parameter could lead to HTTP 500

    Invalid query parameter could lead to HTTP 500, although
    Placement used JSON Schema verification to check input query
    params, but query like: GET allocation_candidates?limit=%88
    will still lead to HTTP 500, as it failed to parse at webob
    which is pre JSON Schema check.

    Change-Id: Iba8d29cb442c610de53e70c81533a8e1243d12dc
    Partial-bug: #1746202

Revision history for this message
Matt Riedemann (mriedem) wrote :

Looks like this is fixed for nova as of https://review.openstack.org/#/c/539164/

tags: added: api
Changed in nova:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.