Comment 17 for bug 605543

I tried it, but as I already feared it lead to:

Gwibber Dispatcher: DEBUG Traceback:
Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/dispatcher.py", line 75, in perform_operation
    message_data = PROTOCOLS[account["protocol"]].Client(account)(opname, **args)
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/twitter.py", line 144, in __call__
    return getattr(self, opname)(**args)
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/twitter.py", line 147, in receive
    return self._get("statuses/home_timeline.json", count=count, since_id=since)
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/twitter.py", line 135, in _get
    if parse: return [getattr(self, "_%s" % parse)(m) for m in data]
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/twitter.py", line 78, in _message
    m = self._common(data)
  File "/usr/lib/python2.6/dist-packages/gwibber/microblog/twitter.py", line 64, in _common
    m["text"] = unescape(data["text"].encode("latin1"))
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-21: ordinal not in range(256)

That modification would make Gwibber unable to support any characters not in latin-1 (such as Chinese, Cyrillic, Arabic, etc...). I really think the problem is with htmllib.py and sgmllib.py, which are unable to handle a mix of unicode and 8-bit strings. Or possibly the problem is that the default encoding on Lucid seems to be utf-8 (although I can't find why), which sgmllib.py doesn't seem to able to handle.