xml_body returns backtrace on XMLSyntaxError

[Tony NIU]

If password contain special character, for example '&', it will raise an exception when request authentication with xml body while authentication successfull with json body.

    $curl -X POST -H Content-Type:application/xml localhost:5000/v2.0/tokens -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?><auth xmlns="http://docs.openstack.org/identity/api/v2.0"><passwordCredentials username="test" password="test321&sdf"/></auth>'

The lxml lib raise an exception.

  File "/opt/stack/keystone/keystone/common/serializer.py", line 63, in __call__
    dom = etree.fromstring(xml_str.strip())
  File "lxml.etree.pyx", line 2754, in lxml.etree.fromstring (src/lxml/lxml.etree.c:54631)
  File "parser.pxi", line 1578, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:82748)
  File "parser.pxi", line 1457, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:81546)
  File "parser.pxi", line 965, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:78216)
  File "parser.pxi", line 569, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:74472)
  File "parser.pxi", line 650, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:75363)
  File "parser.pxi", line 590, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:74696)
XMLSyntaxError: EntityRef: expecting ';', line 1, column 115

    $curl -X POST -H Content-Type:application/json localhost:5000/v2.0/tokens -d '{"auth": {"passwordCredentials": {"username": "test", "password": "test321&sdf"}}}'
Authentication successfull.

[gordon chung (chungg)]

is that a literal ampersand? i tried the xml using an escaped ampersand ('[&]amp[;]') and it worked fine.

according to w3 specs, you must escape ampersand character for it to be valid xml (http://www.w3.org/TR/xml/#syntax):

The ampersand character (&) and the left angle bracket (<) must not appear in their literal form, except when used as markup delimiters, or within a comment, a processing instruction, or a CDATA section. If they are needed elsewhere, they must be escaped using either numeric character references or the strings " &amp; " and " &lt; " respectively. The right angle bracket (>) may be represented using the string " &gt; ", and must, for compatibility, be escaped using either " &gt; " or a character reference when it appears in the string " ]]> " in content, when that string is not marking the end of a CDATA section.

[Tony NIU (niuwl586-v)]

Thank you for your response.
It works when I replace "&" with "&amp;".

[Dolph Mathews (dolph)]

The implementations appears to raise a 500 instead of a 400, and includes a full backtrace in the response. A 400 with message related to XML parsing should be returned instead:

$ curl -v -X POST -H Content-Type:application/xml localhost:5000/v2.0/tokens -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?><auth xmlns="http://docs.openstack.org/identity/api/v2.0"><passwordCredentials username="test" password="test321&sdf"/></auth>'
* About to connect() to localhost port 5000 (#0)
* Trying ::1...
* Connection refused
* Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 5000 (#0)
> POST /v2.0/tokens HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: localhost:5000
> Accept: */*
> Content-Type:application/xml
> Content-Length: 181
>
* upload completely sent off: 181 out of 181 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Content-Length: 5883
< Date: Wed, 06 Mar 2013 20:44:01 GMT
< Connection: close
<
Traceback (most recent call last):
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/eventlet/wsgi.py", line 383, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/Users/dolph/Projects/keystone/keystone/contrib/access/core.py", line 49, in __call__
    response = request.get_response(self.application)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/request.py", line 1296, in send
    application, catch_exc_info=False)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application
    app_iter = application(self.environ, start_response)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
    return resp(environ, start_response)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
    return self.func(req, *args, **kwargs)
  File "/Users/dolph/Projects/keystone/keystone/common/wsgi.py", line 353, in __call__
    response = request.get_response(self.application)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/request.py", line 1296, in send
    application, catch_exc_info=False)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application
    app_iter = application(self.environ, start_response)
  File "/Users/dolph/Environments/os/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
    resp = self.call_func(req, *args, **self.k...

[Dolph Mathews (dolph)]

Just wanted to check in and see if you have a patch for this? We're getting down to the last few bugs blocking RC1, and this is on the list :)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/24441

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/24441
Committed: http://github.com/openstack/keystone/commit/a80a1f8b300cadf69a3ad385cfe0d39a6ad2b9b1
Submitter: Jenkins
Branch: master

commit a80a1f8b300cadf69a3ad385cfe0d39a6ad2b9b1
Author: David Höppner <email address hidden>
Date: Sun Mar 10 20:04:07 2013 +0100

    xml_body returns backtrace on XMLSyntaxError

    Protected against XMLSyntaxError that can occur in from_xml. Return
    a validation error (400) instead of an internal server error (500).

    Change-Id: Ic5160f4f6c810e96b74dbf9563547ac739a54c5e
    Fixes: bug #1101043