Major attachment handling bug!

[Avalon-users]

Attachments are not handled correctly.
Attached to this bug report is an example email in a mbox file.
How to reproduce:
1. create a list called "clasohlson"
2. copy this mbox file to the archive mbox file
3. run bin/arch for the list
4. look at the resulting archive

I will post a followup which will contain the resulting html file I got
when
doing this.

[http://sourceforge.net/tracker/index.php?func=detail&aid=669081&group_id=103&atid=100103]

[Barry Warsaw (barry)]

There was no attachment, are you sure you clicked the
checkbox? :(

[Avalon-users (avalon-users)]

Sorry, my web browser hung after submitting the bug, and I had to go
offfline
in a hurry. Trying again.... :)

[Avalon-users (avalon-users)]

Here is the generated html page for the email, if list language is set to

norwegian.

[Avalon-users (avalon-users)]

And here is the html file in english. Notice the difference. Lots of
garbage (also
called mojibake?)
;)

[Avalon-users (avalon-users)]

-changing priority and assignment-

[Tokio Kikuchi (tkikuchi)]

Please try this patch.
https://sourceforge.net/tracker/?func=detail&atid=300103&aid=670167&group_id=103
(Patch ID 670167)

[Avalon-users (avalon-users)]

Applied without problems, but gave me the following error when running
bin/arch (not on the mbox I submitted, though):

Traceback (most recent call last):
  File "bin/arch", line 187, in ?
    main()
  File "bin/arch", line 175, in main
    archiver.processUnixMailbox(fp, start, end)
  File "/home/mailman/Mailman/Archiver/pipermail.py", line 544,
in
processUnixMailbox
    m = mbox.next()
  File "/usr/lib/python2.2/mailbox.py", line 33, in next
    return self.factory(_Subfile(self.fp, start, stop))
  File "/home/mailman/Mailman/Mailbox.py", line 79, in scrubber
    return mailbox.scrub(msg)
  File "/home/mailman/Mailman/Mailbox.py", line 99, in scrub
    return self._scrubber(self._mlist, msg)
  File "/home/mailman/Mailman/Handlers/Scrubber.py", line 145,
in process
    cs = csre.search(part['content-type'])
TypeError: expected string or buffer

The offending message had no "Content-Type:" field in its
header. (It was,
of course, produced by a Microsoft product... ;)

Some issues from my earlier subimtted example that still needs to be
fixed:
1. The attachment file name still becomes ".dot", not
".doc" as it should
have been. Also, the reported size is wrong (larger) than the actual
file.
2. "The attachment was scrubbed" part in the archive web page
is still sort
of "attached" to the message itself, e.g. to the email signature
of the user
that posted the message.

[Avalon-users (avalon-users)]

Check also the attached "attachment_error.mbox". It contains a
single message,
with a MS Word document that becomes completely scrambled after going
through pipermail...

[Avalon-users (avalon-users)]

Sorry, forget my last comment, the attachment was just fine. My mistake.
But attachment naming is still a problem.

[Tokio Kikuchi (tkikuchi)]

I have updated the patch. Please try.
(Patch ID 670167)

[Avalon-users (avalon-users)]

Applied cleanly, and now there is only one problem left! :D
I believe Mailman is supplying the wrong MIME type for the .doc file when

the archive is private. Thus, the webbrowser just displays the raw file
instead of launching an application that can handle it correctly. I
changed
the archive to public, and the file was displayed correctly.
Thanks for your quick fix so far, this has really helped a lot! (The users
of
the particular list were depending heavily on the archive for the list,
and
considered the list completely unuseable without attachments available in

the archive.)

[Janjules (janjules)]

After applying the patch it still doesn't work. I deleted the directory of
the
archive and rebuild the archive by using bin/arch. But even now, i got the

same: no attachment, only
")Ãj×¢jxZž¢´§Öz|.× zÇ!ç]z»Cj×¢?©žŠÊ<óŸuAœ"±È^žš". Did
i
forget anything to do?

[Barry Warsaw (barry)]

TK, I've looked at the patch, and I like parts of it but not
other parts. For me, get_content_charset() always digs out
the right character set without the need for that regexp.
If it doesn't for you, then that's a bug in the email
package, which we should report and fix.

I've attached my tentative patch which seems to fix the
clasohlson mbox when using Czech. I haven't yet tried
attachment_error.mbox, and I haven't fixed the private
archiver or the mimetypes lie, but will get to those next.

[Barry Warsaw (barry)]

Ok, attachment_error.mbox was shallow. I'll have a fix for
that too.

[Barry Warsaw (barry)]

Ok, try the attached. I haven't checked the private cgi
yet, but I'm sure that's an unrelated problem. This fixes
all the other issues.

[Barry Warsaw (barry)]

I'm going to close this one as fixed. I'll have one or two
minor refinements to check in momentarily.