Mailman can cause extraneous tabs to be displayed in Subject: headers

[Mark Sapiro]

The genesis of this problem is that the header folding and unfolding algorithms specified in RFC822 could introduce extraneous white space in headers which have been folded and unfolded. RFC822 says when folding, "a CRLF immediately followed by AT LEAST one LWSP-char" may be inserted where linear-white-space is allowed, and unfolding "is accomplished by regarding CRLF immediately followed by a LWSP-char as equivalent to the LWSP-char." Thus, one can insert "AT LEAST one LWSP-char" when folding but not remove any when unfolding.

The later RFCs 2822 and 5322 clarify the situation by specifying folding as inserting CRLF preceding existing white-space and unfolding as removing any CRLF which is immediately followed by white-space.

Various MUAs and supporting libraries including the Python 2 email library used by Mailman still follow the RFC822 method of folding and can fold by inserting CRLF followed by TAB and the TAB then doesn't get removed in unfolding.

There is actually control in the Python email library to use TAB or SPACE, and Mailman attempts to determine what character has been used and uses that, but in the case where a Subject: header is not folded but becomes long enough to be folded because of the insertion of a subject_prefix for example, Mailman defaults to a TAB which doesn't get removed in unfolding.

The situation will never be perfect as long as there are MUAs that fold per RFC822 and MUAs that attempt to compensate by removing some white-space following CRLF, but it will be improved somewhat by defaulting to folding with SPACE rather than TAB.