admindb fail to find message charset

[Tokio Kikuchi]

admindb.py displays held message excerpt but fails to get proper message charset, leading unreadable text excerpts especially for languages using multi-byte code.
Attached patch solves this problem by finding the first charset in multipart message, which is used in Decorate.py etc.

[Mark Sapiro (msapiro)]

Tokio,

I'm glad you are having time to work on Mailman again. Welcome back. Your help is valuable, especially with non-western character sets.

Regarding this issue, I see the problem and I see what your patch does, but I wonder if it would be better if rather than looking at the first part with a charset, if we looked at the first main type text part. If it has a charset, the result will be the same either way, but if not, maybe we should use us-ascii rather than the charset of some later part.

I.e.
+ # We get it from the first text part
+ for part in msg.walk():
+ if part.get_content_maintype() == 'text':
+ mcset = part.get_content_charset('us-ascii')
+ break
+ else:
+ mcset = 'us-ascii'

I am thinking of the possibility of a message with an undeclared us-ascii body and an attachment encoded in some other character set. Of course, if the other character set is a superset of us-ascii, it would still be OK to use it, and even in a, hopefully rare, case where it didn't, the moderator could always forward the message to himself to see it, so maybe it doesn't matter much.

What do you think?

[Mark Sapiro (msapiro)]

Indentation is lost in comment 2. It should be

+....# We get it from the first text part
+....for part in msg.walk():
+........if part.get_content_maintype() == 'text':
+............mcset = part.get_content_charset('us-ascii')
+............break
+....else:
+........mcset = 'us-ascii'

[Tokio Kikuchi (tkikuchi)]

Hi, Mark. Your modification to the patch looks reasonable. I should have used get_content_charset() instead of get_param(), at the least.

[Mark Sapiro (msapiro)]

I made one additional change to the patch. Instead of "mcset = part.get_content_charset('us-ascii')", I do "mcset = part.get_content_charset() or 'us-ascii'" as in Decorate.py to allow for charset= with no value.