Find member does not match name in multibyte characters

[KOMEDA Shinji]

findmember encoding is not match menber name encoding.

[Mark Sapiro (msapiro)]

I see the bug, but the patch doesn't fix the problem in all cases. I think a better patch is instead of

        regexp = regexp.decode()

to do

        regexp = regexp.decode(Utils.GetCharSet(mlist.preferred_language)).

I still need to do more testing, but I would like to know if providing this character set in this way still fixes the bug in your environment.

The problem with the original patch is decode() without a charset uses Python's default string encoding which is often ASCII resulting in UnicodeDecodeError and no change to regexp. While this doesn't make things worse, it doesn't fix the problem unless the site has changed Python's default string encoding to a charset more appropriate to the installation.

[KOMEDA Shinji (komeda-shinji)]

I'm using Mailman 2.1.16 in Ubuntu box, in my environment

    DEFAULT_SERVER_LANGUAGE = 'ja'

and Ubuntu has the following code in Mailman/Defaults.py

    import sys
    reload(sys)
    sys.setdefaultencoding('utf-8')

These code came from debian/patches/91_utf8.patch.

I changed "regexp.decode()" to "regexp.decode(Utils.GetCharSet(mlist.preferred_language))",
It works fine.

Thank you.

[Mark Sapiro (msapiro)]

This bug is 'mostly' fixed. There are potentially very complex cases of lists with English as the preferred language (admin UI language) with members whose real names contain non-ascii. In these cases, the search string POSTed by the browser may contain HTML entities such as é and é and parsing all this is complicated and error prone and not done by this fix.

Since the search string is a regexp, one can just use a dot (.) instead of a problem character in these cases.

For lists whose preferred language (admin UI language) is other than English, this bug should be fixed.