zope.exceptions.exceptionformatter and non-ASCII characters

[Marius Gedminas]

I used zope.exceptions.exceptionformatter.format_exception (as_html=False) in an application to show tracebacks in views. I have encountered situations when format_exception returned me a list of UTF-8 strings (my system locale is UTF-8), other situations where I got a list of Unicode strings, and even situations where the list contained a mixture of UTF-8 and Unicode strings (causing UnicodeDecodeErrors in ''.join(...)).

The contract of the exception formatter should explicitly say whether it returns Unicode or not, and various formatting methods should take care to convert the strings as appropriate. I would prefer to get Unicode always.

Source of 8-bit characters: linecache.getline, traceback.format_exception_only. Suspected source of Unicode strings: __traceback_info__/__traceback_supplement__.

There are some calls to str() and, log.Formatter uses cStringIO -- these would break with Unicode strings.

(Side issue: TextExceptionFormatter also sometimes performs HTML escaping.)

[Marius Gedminas (mgedmin)]

Incidentally, there's interesting code in zope.error where it tries to cope with this issue and force everything into ASCII.

[Tres Seaver (tseaver)]

Please follow-up on Github:

  https://github.com/zopefoundation/zope.exceptions/issues/1