Broken PGP/MIME signature of messages with attachments

[Egagnon-users]

Hi,

When I send a PGP/MIME signed message *with*
attachments (in addition to the PGP signature) to a
Mailman-based mailing list, Firefox/Enigmail fails to
verify the signature of the message forwarded by mailman.

I have uploaded an example message with a broken
signature as attachment to this bug.

You can also see how Mailman's automatic archives don't
even handle such messages correctly by looking at how
it archived that message at:

http://sablecc.org/lists/sablecc-user/2004-December/000159.html

I've looked in the bugs database and found that a
somewhat related problem was reported a while ago, but
note that the current bug is different. The original
bug was about PGP/MIME signed messages without
attachments that were handled incorrectly by Enigmail.
 This one is about messages with attachments, and it is
likely that the bug is Mailman's, at least accroding to
the reply I got in:

http://bugzilla.mozdev.org/show_bug.cgi?id=8466

Thanks for your help (and for the magnificient software)!

Etienne

[http://sourceforge.net/tracker/index.php?func=detail&aid=1085651&group_id=103&atid=100103]

[Egagnon-users (egagnon-users)]

Any update?

[Toniw (toniw)]

I can reproduce this bug also.

Here's a diff what happened to me:
diff -u mail.valid mail.invalid

 Content-Disposition: attachment;
filename=configure.in.pre-pkgconfig.patch
-Content-Type: text/x-patch;
name=configure.in.pre-pkgconfig.patch; charset=UTF-8
+Content-Type: text/x-patch;
name=configure.in.pre-pkgconfig.patch;
+ charset=UTF-8
 Content-Transfer-Encoding: base64

And this was _inside_ gpg-signed block.
Mailman should not alter that section at all, right?
X-Mailman-Version: 2.1.4

Yours,
 Toni Willberg <email address hidden>

[Shub-users (shub-users)]

Look at your content filter settings. What MIME-types are you configured

to pass? At the very least, you should be configured to pass the
following:

multipart/mixed
multipart/alternative
text/plain
application/pgp-signature
multipart/signed

[Egagnon-users (egagnon-users)]

I am attaching a screen shot of the content filtering
settings (see the options.png attachment I have added to
this bug report). In short, content filtering is disabled,
so I don't think it is the cause of the problem. :-/

Etienne

[Shevek-users (shevek-users)]

I checked the source, and this appears to be a bug in
Python. It can read and write e-mails, including headers,
and they are interpreted. However, it doesn't preserve
their whitespace, which is of course fatal for pgp signatures.

The bug is filed for python here:
http://sourceforge.net/tracker/index.php?func=detail&aid=968430&group_id=5470&atid=105470

[Egagnon-users (egagnon-users)]

Dear Bas,

Thanks a lot for identifying the source of this bug and
flagging an upstream bug on Python.

Etienne

[Xkr47 (xkr47)]

from python's Message.py:

    def _get_params_preserve(self, failobj, header):
        # Like get_params() but preserves the quoting of values. BAW:
        # should this be part of the public interface?

According to my analysis of the comment, the function in question does
_not_
guarantee to preserve the header as-is, it only preserves _quoting_. Thus,
I
don't think we can call this a bug in python. If there's more official
documentation elsewhere that states that it should preserve everything
as-is,
then ignore this comment altogether :).

I didn't have time to grasp how mailman actually uses that Message class,

but I think mailman should be changed to use functions that don't alter
the
message, and/or re-implement that work according to the requirements of
mailman (or, users of mailman :).

Anyway, an alternative could be to continue to use the Message.py
functions
to parse the values while analyzing the contents of the message, and then

copypaste the original headers as-is after checking.

- xkr47

[Shevek-users (shevek-users)]

That function does indeed not conserve whitespace. E-mail
doesn't say whitespace in headers (or even the message) must
be preserved, so strictly speaking it's not a bug. However,
it is a missing feature, and I think python is the place to
implement it, not mailman.

The point is that signed e-mails with attachments are sent
as two parts: One which is a complete e-mail by itself, and
one which holds the signature for that e-mail. This is
because in attachments whitespace must be preserved (as
opposed to the situation in the message itself).

It would be very convenient if such e-mail can be handled by
python programs without rewriting the things which work fine
for non-signed e-mails. If the attachment is really
changed, of course the signature will be broken. However,
if it is not, it should not be hard to write out the same
message (with whitespace preserved) as was read in. If this
is not possible, it is quite hard to do it in the program,
as python doesn't seem to support loading a part of an
e-mail which has headers of its own as a binary attachment
(without touching whitespace, in particular).

[Proffe (proffe)]

This is a duplicate of bug #815297.

[Sjlongland (sjlongland)]

Originator: NO

I'm noticing this behaviour myself on a mailing list I set up for a group
at uni.

HTML emails signed using PGP/MIME and plaintext emails with attachments,
both fail signature tests when checked in Enigmail 0.94.0.0 (20060110) on
Thunderbird 1.5.0.10 (20070310).

I'm running Mailman 2.1.9_rc1 (Gentoo ebuild) on Linux, with the Qmail
MTA.

[Jan Pokorný (p.o.k.i)]

The same behaviour observed with 2.1.12.
Any idea whether Mailman 3 fixes this issue?

[Mark Sapiro (msapiro)]

In comment #9 above, 815297 is a Sourceforge bug number. The issue in launchpad is https://bugs.launchpad.net/mailman/+bug/265967

Regarding comment #11 and Mailman 3, Mailman 3 uses Python 3 and the email package, which is the source of this issue, has been almost completely rewritten for Python 3. One of the goals of the rewrite was to make the result of parsing/flattening match the original text, so I suspect this is fixed in MM 3, but I'm not certain.