Mailman strips format=flowed from Content-Type header

[Radiantskies]

It seems that Mailman strips the format=flowed
argument from the Content-Type: text/plain header
when delivering incoming e-mails to subscribers.

[http://sourceforge.net/tracker/index.php?func=detail&aid=1495122&group_id=103&atid=100103]

[Radiantskies (radiantskies)]

I've browsed the mailman source code a bit and I found
some suspicious code
in /trunk/mailman/Mailman/Handlers/Decorate.py. If the
list admin specifies a header or footer to be added to
outgoing messages, then the content-type header is zapped
and then regenerated from just the charset info, i.e. the
format=flowed parameter is lost.

[Mark Sapiro (msapiro)]

I think you have found the cause. Scrubber also does the
same thing when flattening the message, but now we have a
difficult decision.

If we want to 'fix' Decorate, we now are faced with a
situation where we may be adding a 'format=fixed' header
and/or footer to a 'format=flowed' message body. This
potential incompatability may require us to 'wrap' the
message and add the header/footer as separate parts in this
case. This will undoubtedly raise additional complaints.

In the case of Scrubber, this may lead us to scrub the
actual message body part - not a good thing.

I may be over-reacting here. It may turn out that it is OK
to render the header/footer/scrubber messages as
'format=flowed'. Although I can contrive a footer for
example (by putting trailing spaces on intermediate lines)
that will be munged by rendering as 'format=flowed', this
may not be a problem in practice, and Decorate could always
be made to strip the spaces.

There are also potentially issues with stripping of leading
spaces because of the assumption that they are 'stuffed'. I
don't know how big an issue this is in practice.

Note, if we do preserve 'format=' we also need to preserve
'delsp='.

[Ptwithy (ptwithy)]

Man this bug drives me NUTS! Surely it is more important to maintain the

content of the message that to quibble about the format on header/footer.

Any halfway intelligent mail client will wrap long URLs, expecting them to
be
reflowed by the recipient, but since mailman strips out the format and
delsp
from the Content-type, the recipient is left in the lurch and sees the URL
as
multiple lines.

In format-flowed, only lines that end in <space><linefeed> are re-flowed.
If
the header/footer lines end in <linefeed> they will not be reflowed, so I
don't
see the need to have a different format for them?

What am I missing?

[Mark Sapiro (msapiro)]

Regarding the previous comment. I don't disagee with you,
and my 2006-06-13 comment wasn't meant to imply that I
wasn't interested in fixing this, just that there might be
more to it than is apparent at first glance. I do have it on
my 'to do' list.

[Ptwithy (ptwithy)]

Thanks, that's great.

For now, I am very happy that I can just turn off my footer and get the
right
behavior (as a work-around).

[Mark Sapiro (msapiro)]

Originator: NO

I have attached a patch (flowed.patch.txt) that applies to Mailman 2.1.9
and 2.1.8. Line numbers may need adjusting for earlier versions.

This is a preliminary patch for testing and shouldn't be considered
official at this point. Feedback will be appreciated.
File Added: flowed.patch.txt

[Mark Sapiro (msapiro)]

Originator: NO

I have replaced the flowed.patch.txt file attached. The previous scrubber
patch didn't work. This one has been more thoroughly tested and does work
at least for the test cases tried. There is an additional fix in this
scrubber patch which has to do with getting the appropriate character set
for the part separator. See the thread beginning at
<http://mail.python.org/pipermail/mailman-users/2006-November/054515.html>.
File Added: flowed.patch.txt

[Mark Sapiro (msapiro)]

Originator: NO

I replaced the patch again. This time, the Scrubber.py patch is
functionally identical to 217616: flowed.patch.txt, but I have added more
comments to the Scrubber patch specifically related to why it won't work
for the archive or the plain format digest.

The Decorate.py patch has been updated to remove any trailing spaces from
lines of msg_header and msg_footer so they won't be inadvertently flowed.

I have the feeling that the biggest issue for most occurs in Decorate.py
in adding msg_header and/or msg_footer, and this is fixed.
File Added: flowed.patch.txt

[Mark Sapiro (msapiro)]

Originator: NO

And yet again I have replaced the patch. Scrubber.py would throw an
exception if a message it was processing had no text/plain part. The new
patch adds line 190 to Scrubber.py

    format = delsp = None
File Added: flowed.patch.txt

[Mark Sapiro (msapiro)]

This was fixed in Mailman 2.1.10.