PROPFIND with a unicode ID fails

[Mark Hammond]

I've attached a simple little test script. It does a PUT, then a GET, then a PROPFIND for a unicode URL (encoded as UTF8).

The PUT and GET work correctly. The PROPFIND fails - but then yields a UnicodeEncodeError rendering the error message. The traceback from the trunk is below (the traceback from the webdav branch is very similar, but a few method names have changed. I'm also attaching the test script.

Traceback (most recent call last):
  Module zope.publisher.publish, line 138, in publish
    result = publication.callObject(request, object)
  Module zope.app.publication.http, line 73, in callObject
    return mapply(ob, request.getPositionalArguments(), request)
  Module zope.publisher.publish, line 113, in mapply
    return debug_call(object, args)
   - __traceback_info__: <bound method PROPFIND.PROPFIND of <zope.app.dav.propfind.PROPFIND object at 0x0394D4B0>>
  Module zope.publisher.publish, line 119, in debug_call
    return object(*args)
  Module zope.app.dav.propfind, line 102, in PROPFIND
    self._handlePropvalues(None, resp)
  Module zope.app.dav.propfind, line 177, in _handlePropvalues
    self._renderAvail(avail, resp, _props)
  Module zope.app.dav.propfind, line 259, in _renderAvail
    value = widget()
  Module zope.app.dav.widget, line 44, in __call__
    return str(self)
  Module zope.app.dav.widget, line 41, in __str__
    return str(self._data)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position 10: ordinal not in range(128)

[Stephan Richter (srichter)]

Changes: submitter email, importance (medium => critical)

[Martijn Faassen (faassen)]

I can still reproduce a problem with unicode, though the traceback has changed in 3.3 branch:

  File "/home/faassen/working/Zope-3.3/src/twisted/web2/wsgi.py", line 139, in run
    result = self.application(self.environment, self.startWSGIResponse)
  File "/home/faassen/working/Zope-3.3/src/zope/app/wsgi/__init__.py", line 55, in __call__
    request = publish(request, handle_errors=handle_errors)
  File "/home/faassen/working/Zope-3.3/src/zope/publisher/publish.py", line 141, in publish
    publication.handleException(
  File "/home/faassen/working/Zope-3.3/src/zope/app/publication/zopepublication.py", line 355, in handleException
    response.handleException(exc_info)
  File "/home/faassen/working/Zope-3.3/src/zope/publisher/http.py", line 900, in handleException
    self.setResult(body)
  File "/home/faassen/working/Zope-3.3/src/zope/publisher/http.py", line 817, in setResult
    body, headers = self._implicitResult(result)
  File "/home/faassen/working/Zope-3.3/src/zope/publisher/http.py", line 860, in _implicitResult
    raise ValueError(
exceptions.ValueError: Unicode results must have a text content type.

[Martijn Faassen (faassen)]

I found out that Zope 3 is actually *hiding* the true error now for some reason. The error still remains visible at the level of zope.app.zopepublication, but then doesn't get displayed properly,
possibly because the content-type is set to None by the
time it enters _implicitResult in zope.publisher.http

[Martijn Faassen (faassen)]

Some more debugging indicates that it's likely due to the Content-Type header being None. I've investigated the history of zope.publisher.http, and can't see any relevant changes there since the beginning of the year. Still evidently when the bug was reported in march, the trunk did set Content-Type to something so the error could reported properly, but it has stopped doing so now.

[Martijn Faassen (faassen)]

Further research shows that the error *does* get reported correctly
with the error reporting utility, but does not get reported correctly on the command line.

[Martijn Faassen (faassen)]

I just found out that there is already an issue describing the error handling problem, so I'll continue discussion there:

http://www.zope.org/Collectors/Zope3-dev/686

[Dmitry Vasiliev (hdima)]

I'll look into this. There is also possible problem with SequenceDAVWidget.__str__() which try to return Unicode.

[Dmitry Vasiliev (hdima)]

Fixed on the trunk at revision 74705

[Philipp von Weitershausen (philikon)]

We've had two betas since this fix has been committed, therefore setting status to Fix Released.