mailman rejects encoded from addresses

[Curtis Hovey]

A user subscribed to a launchpad list reported his email were not being received. This issue may be a duplicate of bug 702516, but the patch I read does not appear to be working with the encoding issue. This is what user Lp user ~ralf-claussnitzer provided:

We managed to figure out why LP is rejecting my mail. Shortcut: my name contains special characters. LP is not able to identify my email address form the mail header.

Here is a detailed explanation:

In failure cases my name "Ralf Claußnitzer" is encoded in the From field like this:

(given it's sent using ISO)
From: =?iso-8859-1?Q?Clau=DFnitzer=2C_Ralf?= <email address hidden>

(given it's sent using UTF-8)
From: =?utf-8?B?Q2xhdcOfbml0emVyLCBSYWxm?= <email address hidden>

If the special character "ß" is removed from the mail servers dictionary, the From-Field looks more like expected and LP delivers that mail:

(using UTF-8 but without special character)
From: "Claussnitzer, Ralf" <email address hidden>

Sending messages with HTML body is not a problem for LP.

As LP decides whether to moderate or block a message based on the From field, it looks like LP is not able to identify the address from those screwed up headers.

However, RFC5335 on Internationalized Email Headers states on Page 5 (http://tools.ietf.org/html/rfc5335#page-5) that "The bodies of header fields are allowed to contain UTF-8 characters, but the header field names themselves must contain only ASCII characters." If so, I don’t understand where the encoding shown above comes from. But I expect LP to handle those messages correctly if a valid email address is in the From field somewhere.

[Mark Sapiro (msapiro)]

The remark about RFC5335 is not relevant. The example From: headers in the report are RFC 2047 encoded, and are valid as the encoded headers contain only us-ascii characters.

In any case, the report talks about LP rejecting mail. Is this mail to a Mailman list? If so, what happens to the mail. Is there an actual rejection notice? If so, what does it say? What's in Mailman's various logs, in particular 'vette' and 'error', relating to these messages? What are the lists settings for Privacy options... -> Sender filters? Is this user in accept_these_nonmembers? What is generic_nonmember_action?

This very well could be bug 702516 as the actual decoded From: header is

From: Claußnitzer, Ralf <email address hidden>

which does contain an unquoted comma which is the issue of bug 702516. If this is the issue, the reason it works if the "ß" is replaced with "s" is the real name then contains no non-ascii, so the users MUA simply quotes it rather than RFC 2047 encoding it.

[Curtis Hovey (sinzui)]

This user was unaware that sending emails to a launchpad list does go directly the Launchpad's mailman. The mail was not forwarded to the subscribers. It was not archived. It was not moderated because of Launchpad's own handler. Launchpad discards any message that does not come from a Launchpad user.
    http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/services/mailman/monkeypatches/lphandler.py

The crucial issue is that
    msg.get_sender()
is not returning the email address (<email address hidden>). WIthout an exact match, Lp will tell mailman to discard the message.

Thank you for a second reading of the encoded name. I neglected to see that the name order was reversed. I agree that this is a dupe of bug 702516.

[Mark Sapiro (msapiro)]

See bug 702516 for fix.