Shunt clogged up - RFC2231 unicode problem?

[Ventoux]

Dozens of messages are backed up in my shunt directory
with the message "TypeError: unicode() argument 2 must
be string, not None" (full traceback example below).

This seems very similar to a problem fixed by ber with 2.0.x
earlier this year but I'm running 2.1.2 (on FreeBSD). I tried
my best to apply a similar solution to Message.py but
apparently I don't know enough about python to make it
work.

Here's a typical Traceback --

Aug 16 16:08:25 2003 (46338) Uncaught runner
exception: unicode() argument 2 must be string, not None
Aug 16 16:08:25 2003 (46338) Traceback (most recent
call last):
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line
105, in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/Runner.py", line
155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Queue/
IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/local/mailman/Mailman/Queue/
IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py",
line 89, in process
    send_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py",
line 130, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/usr/local/mailman/Mailman/Handlers/ToDigest.py",
line 303, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/usr/local/mailman/Mailman/Handlers/Scrubber.py",
line 257, in process
    url = save_attachment(mlist, part, dir)
  File "/usr/local/mailman/Mailman/Handlers/Scrubber.py",
line 348, in save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
  File "/usr/local/mailman/pythonlib/email/Message.py", line
707, in get_filename
    return unicode(newvalue[2], newvalue[0])
TypeError: unicode() argument 2 must be string, not None

[http://sourceforge.net/tracker/index.php?func=detail&aid=789897&group_id=103&atid=100103]

[Ventoux (ventoux)]

ps - moving the various digest.mboxes freed up the blocked lists -
too many messages with attachments in the digests for me to
figure out exactly which one was causing the problem, but I'll be
happy to forward copy of offending mbox to anyone interested.

[Barry Warsaw (barry)]

A better thing to do is to attach a .pck/.db pair of files
for a shunted message. Then I can run it through my system
and hopefully reproduce the bug.

If the message contains sensitive information or you don't
want to upload it for other reasons, please email them as
attachments to me: <email address hidden>. Make sure you put
"Mailman" in the subject.

[Barry Warsaw (barry)]

This was caused by a bug in the email package. It'll be
fixed in email 2.5.4 and in Mailman 2.1.3.

[Jared Mauch (jared-0)]

I'm seeing what appears to be a similar error related to
unicode.

i can send you .db and .pck files if it would help.
mine is with mailman-2.1.2

Aug 27 01:12:03 2003 (24565) Uncaught runner exception:
ASCII encoding error: ordinal not in range(128)
Aug 27 01:12:03 2003 (24565) Traceback (most recent call last):
  File "/home/mailman/Mailman/Queue/Runner.py", line 105, in
_oneloop
    self._onefile(msg, msgdata)
  File "/home/mailman/Mailman/Queue/Runner.py", line 155, in
_onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/home/mailman/Mailman/Queue/IncomingRunner.py", line
130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/home/mailman/Mailman/Queue/IncomingRunner.py", line
153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/home/mailman/Mailman/Handlers/ToDigest.py", line
89, in process
    send_digests(mlist, mboxfp)
  File "/home/mailman/Mailman/Handlers/ToDigest.py", line
130, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/home/mailman/Mailman/Handlers/ToDigest.py", line
303, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/home/mailman/Mailman/Handlers/Scrubber.py", line
305, in process
    t = unicode(t, 'ascii', 'replace').encode('ascii')
UnicodeError: ASCII encoding error: ordinal not in range(128)

Aug 27 01:12:03 2003 (24565) SHUNTING:
1061961121.185183+0b73a771780f550a6c2e25855f37ea5f250ed212

[Barry Warsaw (barry)]

Please wait until 2.1.3 is released (whenever I can crawl
out of SoBig consequences). If the bug persists, please
submit a new one then.

[Joaofraga (joaofraga)]

I am experiencing this bug with Mailman 2.1.4 and Python
2.3.3.
Here goes a relevant part of the error log:

Jan 15 02:02:58 2004 (25537) Uncaught runner exception:
unicode() argument 2 must be string, not None
Jan 15 02:02:58 2004 (25537) Traceback (most recent call last):
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 105,
in _oneloop
    self._onefile(msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 155,
in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py",
line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/usr/lib/mailman/Mailman/Queue/IncomingRunner.py",
line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line
89, in process
    send_digests(mlist, mboxfp)
  File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line
130, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/usr/lib/mailman/Mailman/Handlers/ToDigest.py", line
303, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/usr/lib/mailman/Mailman/Handlers/Scrubber.py", line
257, in process
    url = save_attachment(mlist, part, dir)
  File "/usr/lib/mailman/Mailman/Handlers/Scrubber.py", line
348, in save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
  File "/usr/lib/mailman/pythonlib/email/Message.py", line
707, in get_filename
    return unicode(newvalue[2], newvalue[0])
TypeError: unicode() argument 2 must be string, not None

Jan 15 02:02:58 2004 (25537) SHUNTING:
1074132178.1086111+ae4e7452c810775c6904c8a4e8c71431dbc4944d