Should return 404 on resource not found

[Feilong Wang]

Based on current implement, InvalidInput, MissingArgument and UnknownArgument of wsme.exc are using both " and ' in the error message. But when the Debug level of CM is set as True, the return code is correct. But if the Debug level is set as False, the error code of InvalidInput, MissingArgument and UnknownArgument will return 500 instead of 400. The error trace is as below:

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/opt/stack/ceilometer/ceilometer/api/app.py", line 105, in __call__
    return self.v2(environ, start_response)
  File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 545, in __call__
    return self.app(env, start_response)
  File "/usr/local/lib/python2.7/dist-packages/pecan/middleware/recursive.py", line 56, in __call__
    return self.application(environ, start_response)
  File "/opt/stack/ceilometer/ceilometer/api/middleware.py", line 75, in __call__
    app_iter = self.app(environ, replacement_start_response)
  File "/usr/local/lib/python2.7/dist-packages/pecan/core.py", line 543, in __call__
    self.handle_request()
  File "/usr/local/lib/python2.7/dist-packages/pecan/core.py", line 501, in handle_request
    result = self.render(template, result)
  File "/usr/local/lib/python2.7/dist-packages/pecan/core.py", line 361, in render
    return renderer.render(template, namespace)
  File "/usr/local/lib/python2.7/dist-packages/wsmeext/pecan.py", line 21, in render
    return wsme.rest.json.encode_error(None, namespace)
  File "/usr/local/lib/python2.7/dist-packages/wsme/rest/json.py", line 237, in encode_error
    return json.dumps(errordetail)
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 321, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 237, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 311, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 213, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: u'Unknown argument: "arg": fake' is not JSON serializable
127.0.0.1 - - [30/Aug/2013 15:38:54] "GET /v2/resources/fake HTTP/1.1" 500 59

Based on my investigation, I found current the method dumps of wsme/json.py can't parse dict like below, no matter the Debug level is True or False.

{'debuginfo': None, 'faultcode': 'Client', 'faultstring': u'Unknown argument: "arg": fake'}

But it works for the standard json lib. So it means the TypeError caused the 500.

However, it still can't explain why the Debug level will impact the return code.

[gordon chung (chungg)]

i'm not sure i follow this. does this mean depending on whether we use a single quotation or double quotation in the error message, it will throw a different error code?

[Feilong Wang (flwang)]

Yep, you can recreate it by set the Debug level as False. then try command:

ceilometer resource-show -r AAA

it will return 500 instead of 400.

I have discussed it with dhellmann yesterday. But I didn't get any clue based on current findings.

[Doug Hellmann (doug-hellmann)]

It sounds like the code that is trying to produce the JSON version of the error message is failing because it is given an object it does not know how to convert to JSON (the TypeError is "is not JSON serializable"). What object is that? Is it an exception of an unknown type? Perhaps WSME just needs to be smart enough to turn those into unicode() before trying to serialize them?

[Feilong Wang (flwang)]

Not really, just like I said in the #1 comment, you can try:

json.dumps({'debuginfo': None, 'faultcode': 'Client', 'faultstring': u'Unknown argument: "arg": fake'}) with your default json lib on your env. It works great.

But the wsme/json.py can't parse it.

[Doug Hellmann (doug-hellmann)]

What part of the traceback makes you think the problem is in parsing the json? I see calls into the encoder, but not the parser.

[Feilong Wang (flwang)]

oh, sorry, maybe the word "parse" is confusing. I just say the method dumps of wsme/json.py will fail when the parameter is mixing ' and ""

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/47775

[Julien Danjou (jdanjou)]

I've tried reproducing it but I was unable to do it. OTOH, I think we should fix this to return a 404 rather than a 400 in Ceilometer.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/48636

[Julien Danjou (jdanjou)]

I've found the problem, I think it comes from the Ceilometer error middleware. Working on it.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/48636
Committed: http://github.com/openstack/ceilometer/commit/acb89bd2c257fdd2e37bb2e08eb16e89f84a4bbb
Submitter: Jenkins
Branch: master

commit acb89bd2c257fdd2e37bb2e08eb16e89f84a4bbb
Author: Julien Danjou <email address hidden>
Date: Fri Sep 27 14:12:20 2013 +0200

    api: return 404 if a resource is not found

    This also checks for the error message that is returned, and fixes a
    problem with the error encoding middleware that was doing double JSON
    encoding.

    Change-Id: Ieb39a991ddc9ecba0a7e71450a1e57ede18ccbe6
    Fixes-Bug: #1218760
    Fixes-Bug: #1208552