SchoolTool hangs when username includes german Umlaut

Bug #397610 reported by Martin Schimmels on 2009-07-09
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Gediminas Paulauskas

Bug Description

Users that have non-ascii symbols in username cannot login, they get a traceback:

  UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 12: ordinal not in range(128)

  File "/usr/lib/python2.5/site-packages/zope/publisher/", line 130, in publish
  obj = request.traverse(obj)
File "/usr/lib/python2.5/site-packages/zope/publisher/", line 513, in traverse
  ob = super(BrowserRequest, self).traverse(obj)
File "/usr/lib/python2.5/site-packages/zope/publisher/", line 464, in traverse
  ob = super(HTTPRequest, self).traverse(obj)
File "/usr/lib/python2.5/site-packages/zope/publisher/", line 252, in traverse
  publication.callTraversalHooks(self, obj)
File "/usr/lib/python2.5/site-packages/zope/app/publication/", line 129, in callTraversalHooks
  self._maybePlacefullyAuthenticate(request, ob)
File "/usr/lib/python2.5/site-packages/zope/app/publication/", line 123, in _maybePlacefullyAuthenticate
File "/usr/lib/python2.5/site-packages/zope/publisher/", line 516, in setPrincipal
  message = logging_info.getLogMessage()
File "/usr/lib/python2.5/site-packages/zope/app/security/", line 29, in getLogMessage
  return str(

Changed in schooltool:
assignee: nobody → Justas Sadzevičius (justas-pov)
importance: Undecided → Critical
milestone: none → august-1

Thank you for the bug report.

We will prohibit usernames with non-ascii codec in the next release.

Changed in schooltool:
status: New → Confirmed
Tom Hoffman (tom-hoffman) wrote :

So this is a Zope thing, not a SchoolTool thing?

Douglas Cerna (replaceafill) wrote :

When SchoolTool creates the Principal object in it uses:

principal = Principal(id, person.title, person=ProxyFactory(person))

where id is a unicode like u'sb.person.Zur\xfcckverfolgung'. The Principal class tries to encode this using ASCII and it fails. If I change the previous line to:

principal = Principal(id.encode("punycode"), person.title, person=ProxyFactory(person))

I can login as Zurückverfolgung. I don't know the repercussions on functionality if you change the principal id like this, but I like unicode usernames :)

Tom Hoffman (tom-hoffman) wrote :

If we can test and implement this with a reasonably amount of work, we should. If SchoolTool is successful we'll have a *lot* of schools with entirely non-ASCII character sets. We put a lot of work into making SchoolTool work under those conditions, so it would be a shame to drop the ball on something as basic as person ID's.

Changed in schooltool:
milestone: none → karmic
Changed in schooltool:
importance: Critical → High
Changed in schooltool:
milestone: lucid → 1.4.0
status: Confirmed → Triaged
Changed in schooltool:
status: Triaged → Fix Committed
assignee: Justas Sadzevičius (justas-pov) → Douglas Cerna (replaceafill)
Gediminas Paulauskas (menesis) wrote :

For 1.4.0 a check was added that prevents adding users with non-ascii usernames.

Existing users will not be able to login as before, that will have to be fixed later.

Changed in schooltool:
status: Fix Committed → Triaged
milestone: 1.4.0 → none
tags: added: tagged-badly unicode
tags: added: traceback
removed: tagged-badly
Gediminas Paulauskas (menesis) wrote :

We might have issues with unicode usernames, I have fixed two such bugs (Bug #495872 and

But this error, that prevents such users from loggin in, is a bug #131460 in zope.publisher that will be fixed in 3.12.4 (unreleased). When that is released, use it in development schooltool, remove the usernames restriction, and look for more bugs.

Douglas' comment #3 suggests storing an encoded principal id. The fix in zope.publisher is similar, except that it uses'ascii', 'backslashreplace') instead of id.encode("punycode") and does that only for log messages, id is stored as unicode.

description: updated
Changed in schooltool:
milestone: none → next
Changed in schooltool:
assignee: Douglas Cerna (replaceafill) → Justas Sadzevičius (justas-pov)
milestone: next → 1.5.2
status: Triaged → Fix Committed
status: Fix Committed → In Progress

One more traceback, when a student ąžuolas looks at the Manage view (Unauthorized).

2010-09-21 16:10:44,062 Exception while rendering view on exception
Traceback (most recent call last):
  File "/home/justas/.buildout/eggs/", line 379, in handleException
  File "/home/justas/.buildout/eggs/transaction-1.0.0-py2.6.egg/transaction/", line 93, in commit
    return self.get().commit()
  File "/home/justas/.buildout/eggs/transaction-1.0.0-py2.6.egg/transaction/", line 322, in commit
  File "/home/justas/.buildout/eggs/transaction-1.0.0-py2.6.egg/transaction/", line 419, in _commitResources
  File "/home/justas/.buildout/eggs/ZODB3-3.9.4-py2.6-linux-i686.egg/ZODB/", line 767, in tpc_vote
    s = vote(transaction)
  File "/home/justas/.buildout/eggs/ZODB3-3.9.4-py2.6-linux-i686.egg/ZODB/FileStorage/", line 724, in tpc_vote
  File "/home/justas/.buildout/eggs/ZODB3-3.9.4-py2.6-linux-i686.egg/ZODB/FileStorage/", line 278, in asString
    return "".join(map(str, [s, self.user, self.descr, self.ext]))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-13: ordinal not in range(128)

Changed in schooltool:
milestone: 1.5.2 → next
Gediminas Paulauskas (menesis) wrote :

The same exception as comment#7 if the user saves anything, e.g. calendar preferences.

I would say this is a bug in ZODB3, the asString function (last line of traceback).

Gediminas Paulauskas (menesis) wrote :

I have reported this ZODB3 issue as Bug #646019.

Changed in schooltool:
status: In Progress → Triaged
Gediminas Paulauskas (menesis) wrote :

We have to remove these ascii checks after Bug #646019 is fixed. Then test again that there are no new bugs related to unicode usernames.

Changed in schooltool:
assignee: Justas Sadzevičius (justas-pov) → Gediminas Paulauskas (menesis)
Changed in schooltool:
milestone: next → none
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers