Journal crashes with non-ascii usernames

[Marc Woog]

When in calendar view (from whichever person) trying to click on "Journal" in a calender entry results in the error below.

I imported new students, added them to groups, to a section and tried to access their journal (as manager or as teacher).

Exception

  KeyError: u'\xe4'
Traceback

  File "/usr/lib/python2.5/site-packages/zope/publisher/publish.py", line 133, in publish
  result = publication.callObject(request, obj)
File "/usr/lib/python2.5/site-packages/zope/app/publication/zopepublication.py", line 167, in callObject
  return mapply(ob, request.getPositionalArguments(), request)
File "/usr/lib/python2.5/site-packages/zope/publisher/publish.py", line 108, in mapply
  return debug_call(obj, args)
Extra information: <bound method LyceumSectionJournalView.__call__ of <zope.app.publisher.browser.viewmeta.LyceumSectionJournalView object at 0xd74782c>>
File "/usr/lib/python2.5/site-packages/zope/publisher/publish.py", line 114, in debug_call
  return obj(*args)
File "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/journal.py", line 346, in __call__
  return self.template()
File "/usr/lib/python2.5/site-packages/zope/app/pagetemplate/viewpagetemplatefile.py", line 83, in __call__
  return self.im_func(im_self, *args, **kw)
File "/usr/lib/python2.5/site-packages/zope/app/pagetemplate/viewpagetemplatefile.py", line 51, in __call__
  sourceAnnotations=getattr(debug_flags, 'sourceAnnotations', 0),
File "/usr/lib/python2.5/site-packages/zope/pagetemplate/pagetemplate.py", line 115, in pt_render
  strictinsert=0, sourceAnnotations=sourceAnnotations)()
Template "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/templates/journal.pt"
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 271, in __call__
  self.interpret(self.program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 891, in do_useMacro
  self.interpret(macro)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 536, in do_optTag_tal
  self.do_optTag(stuff)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 521, in do_optTag
  return self.no_tag(start, program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 516, in no_tag
  self.interpret(program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 957, in do_defineSlot
  self.interpret(block)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 536, in do_optTag_tal
  self.do_optTag(stuff)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 521, in do_optTag
  return self.no_tag(start, program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 516, in no_tag
  self.interpret(program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 949, in do_defineSlot
  self.interpret(slot)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 536, in do_optTag_tal
  self.do_optTag(stuff)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 521, in do_optTag
  return self.no_tag(start, program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 516, in no_tag
  self.interpret(program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 534, in do_optTag_tal
  self.no_tag(stuff[-2], stuff[-1])
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 516, in no_tag
  self.interpret(program)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 346, in interpret
  handlers[opcode](self, args)
File "/usr/lib/python2.5/site-packages/zope/tal/talinterpreter.py", line 745, in do_insertStructure_tal
  structure = self.engine.evaluateStructure(expr)
File "/usr/lib/python2.5/site-packages/zope/tales/tales.py", line 696, in evaluate
  return expression(self)
Template "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/templates/journal.pt", line 66, column 6
  Expression: <PathExpr standard:u'view/gradebook/render'>
File "/usr/lib/python2.5/site-packages/zope/tales/expressions.py", line 217, in __call__
  return self._eval(econtext)
File "/usr/lib/python2.5/site-packages/zope/tales/expressions.py", line 211, in _eval
  return ob()
File "/usr/lib/python2.5/site-packages/schooltool/table/table.py", line 416, in render
  return formatter()
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 67, in __call__
  self._getCSSClass('table'), self.renderContents(),
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 77, in renderContents
  self.renderRows())
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 455, in renderRows
  return super(AlternatingRowFormatterMixin, self).renderRows()
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 98, in renderRows
  return ''.join([self.renderRow(item) for item in self.getItems()])
File "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/table.py", line 98, in renderRow
  return self._renderRow(item)
File "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/table.py", line 84, in _renderRow
  return super(SelectableRowTableFormatter, self).renderRow(item)
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 464, in renderRow
  self.renderCells(item))
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 111, in renderCells
  [self.renderCell(item, col) for col in self.visible_columns])
File "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/table.py", line 66, in renderCell
  return super(SelectableRowTableFormatter, self).renderCell(item, column)
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 115, in renderCell
  self._getCSSClass('td'), self.getCell(item, column),)
File "/usr/lib/python2.5/site-packages/zc/table/table.py", line 121, in getCell
  return column.renderCell(item, self)
File "/usr/lib/python2.5/site-packages/zc/table/column.py", line 108, in renderCell
  value = self.getter(item, formatter)
File "/usr/lib/python2.5/site-packages/schooltool/lyceum/journal/browser/journal.py", line 106, in getter
  urllib.quote(item.__name__))
File "/usr/lib/python2.5/urllib.py", line 1205, in quote
  res = map(safe_map.__getitem__, s)

[Douglas Cerna (replaceafill)]

I guess some of your usernames have non-ascii characters in them, right? Specifically a "ä" character? SchoolTool doesn't handle non-ascii usernames very well.

The problem in the code is in the schooltool.lyceum.journal.browser.journal.StudentNumberColumn class, in the getter method :

- urllib.quote(item.__name__))
+ urllib.quote(item.__name__.encode("utf-8")))

If you fix this, you get a new and similar error in the schooltool.lyceum.journal.browser.table.SelectStudentCellFormatter class, in its __call__ method:

- urllib.urlencode([('student', item.__name__)] +
+ urllib.urlencode([('student', item.__name__.encode("utf-8"))] +

Then you're able to look at the journal, but you cannot save grades for the students with non-ascii usernames :(

[Marc Woog (mwoog)]

Thanks Douglas! That was the issue. I removed the umlaut from the name and everything worked again. Cheers, Marc

[Tom Hoffman (tom-hoffman)]

Remind me of why we don't handle non-ascii user id's again? Is this a Zope/ZODB limitation? Sloppy coding on our part?

[Gediminas Paulauskas (menesis)]

There is a Bug #397610 that is also caused by German umlaut in a username, but with a different backtrace.

[Gediminas Paulauskas (menesis)]

urllib.quote in the function that fails is not needed, should be cgi.escape. But then you will get another exception because there are other places in journal that don't encode or escape unicode values correctly. This will need a careful review.

Meanwhile, avoid usernames with non-ascii symbols.

[Gediminas Paulauskas (menesis)]

I fixed this crash by changing urllib.quote(item.__name__) to quoteattr(item.__name__).
Grades are saved now.