zope.sendmail: TypeError: character mapping must return integer, None or unicode

[volker.jaenisch]

Due to a change in python2.6 smtplib

[see http://bugs.python.org/issue8489]

it is no longer possible to send unicode login credential to the mailer,
since smtplib expect plain strings or utf-8 codet strings.

This simple patch solves the problem

zope.sendmail-3.7.1-py2.6.egg/zope/sendmail$ diff mailer.py mailer.py~
67c67
< connection.login(self.username.encode('utf-8'), self.password.encode('utf-8'))
---
> connection.login(self.username, self.password)

Best Regards

Volker Jaenisch

[Christian Theune (ctheune)]

Please do not file bugs with Zope 3 anymore, but assign them to the individual packages.

[volker.jaenisch (volker-jaenisch)]

Sorry!

Christian Theune schrieb:
> Please do not file bugs with Zope 3 anymore, but assign them to the
> individual packages.
>
> ** Project changed: zope3 => zope.sendmail
>
Will not happen again.

Best Regards,

Volker

--
====================================================
   inqbus it-consulting +49 ( 341 ) 5643800
   Dr. Volker Jaenisch http://www.inqbus.de
   Herloßsohnstr. 12 0 4 1 5 5 Leipzig
   N O T - F Ä L L E +49 ( 170 ) 3113748
====================================================

[Martijn Pieters (mjpieters)]

The fact that smtplib accepted unicode before was more of an accident than a feature, something that changed with an optimisation in the hmac module, see http://bugs.python.org/issue5285.

Patch looks correct to me. However, can someone find out if UTF-8 makes any sense here. I see a Django newsletter product encode to utf8 here (see http://github.com/Fantomas42/emencia-django-newsletter/issues/issue/1 for example), and so does Trac (http://trac.edgewall.org/ticket/8083). I'd like to see some confirmation though that SMTP servers accept non-ascii characters in usernames or passwords in UTF-8, or if there are any other encodings that need to be supported.

When the answer is that only ASCII is supported, I'd say UTF-8 is just fine here as it protects the user that enters a username and password that won't work anyway will not get a strange encoding error, just failure to send email (as expected).

[Martijn Pieters (mjpieters)]

http://tools.ietf.org/html/draft-ietf-sasl-crammd5-10 applies here. It's a draft specification to update the CRAM MD5 SASL standard to define how to deal with non-ASCII usernames and passwords. It explicitly states usernames and passwords should be encoded to UTF-8.

The python HMAC library change only exposed the bug that smtplib accepted unicode strings for the username and password, and the CRAM MD5 mechanism is not the only authentication method. Although the other methods continue to work with unicode values, they are just as broken in practice. One of the other methods is PLAIN SASL (http://www.ietf.org/rfc/rfc4616.txt), and it already states the encoding used is UTF-8.

So, to summarize my findings: Sending in unencoded Unicode strings is broken for non-ASCII usernames and passwords already. Encoding them is the correct fix, and UTF-8 is the correct encoding for PLAIN, and for the proposed update for CRAM MD5. As any non-ASCII usernames and passwords are broken now, the patch won't make matters worse for those cases.

I'll apply the patch.

[Martijn Pieters (mjpieters)]

For the record, the 3rd and last supported SASL mechanism is LOGIN (http://sepp.oetiker.ch/sasl-2.1.19-ds/draft-murchison-sasl-login-00.txt) and it's specification also states that UTF-8 is the correct encoding.

Marking as assigned to me.

[Martijn Pieters (mjpieters)]

I fixed the problem slightly differently, to deal with the case where username and password are already encoded; utf-8 encoding is only applied when the parameter in question is a unicode string.

Fix committed in r117102 for trunk, r117103 for the 3.7 branch.

[Martijn Pieters (mjpieters)]

Hanno released zope.sendmail 3.7.4 which includes this fix.

[volker.jaenisch (volker-jaenisch)]

Perfect!

Thank you a lot.

Volker Jaenisch

Am 01.10.2010 11:39, schrieb Martijn Pieters:
> Hanno released zope.sendmail 3.7.4 which includes this fix.
>
> ** Changed in: zope.sendmail
> Status: Fix Committed => Fix Released
>
>

--
====================================================
   inqbus it-consulting +49 ( 341 ) 5643800
   Dr. Volker Jaenisch http://www.inqbus.de
   Herloßsohnstr. 12 0 4 1 5 5 Leipzig
   N O T - F Ä L L E +49 ( 170 ) 3113748
====================================================