cc modification due to nodup setting breaks DKIM & thus DMARC

[Ian Kelling]

Keeping DKIM signature intact allows passing DMARC without from munging
(I have references all about this if needed). However, mailman modifies
the cc header when a subscriber has the nodup setting and someone sends
to the list with that subscriber cced. Mailman doesn't email them and
removes them from the cc, presumably so for further replies mailman can
email them without it being a duplicate and they get all the good
mailman headers instead of a plain email from the cc. Anyways, best if
Mailman just not modify the cc if we are in the situation where it
matters for passing DMARC. The attached patch does that.

Note, I'm sending this as an employee of the Free Software Foundation
and my copyright is already going to them.

[Eric Blake (eblake)]

I have long argued that there are two orthogonal things, currently both being managed by the single Avoid Duplicates settings. One is good, one is horrible. I would really love to have TWO separate knobs so that users can choose which of the two features they want.

1. Suppress duplicates - if a user does not want the same mail received twice through two paths, the list should not mail that user when their name is on a to or cc. This feature is good.

2. Rewrite CC - when the list suppresses sending to a given user, it rewrites mail headers to drop that user from cc, on the grounds that anyone replying to the list will still get their reply to the original author (through the list, rather than through cc). This feature is bad.

I _really_ want a way to enable JUST feature 1, and NOT feature 2, for the mailman lists I am subscribed to.

Here's several scenarios where I find feature 2 to be a misfeature, where User A is a user with the feature enabled, users B and C are list subscribers, and user D is not a list subscriber.

Scenario 1: User B writes to the list and to user A in cc (to catch user A's attention). User A gets a copy of the email where they are on CC, but user B and user C get a copy of the email where user A is not on CC. If user C replies to all, the reply will reach user A via the list, but will NOT have user A in cc, failing to catch user A's attention on the followup.

Scenario 2: User D writes to the list and to user A in cc (unaware that user A is subscribed to the list, and trying to catch user A's attention). User A gets the copy of the email where they are on cc, but user B and user C do not see that user A was in CC. Furthermore, user B replies all to the list and to user D, and now user D sees that user A was dropped from the CC. User D now wonders why user B dropped user A from the conversation.

Avoiding the munging of CC when DMARC rules are involved is nice, but does not go as far as I want, which is to have a knob for an individual user to ALWAYS avoid the munging of their own email, regardless of whether duplicate delivery is suppressed, and regardless of DMARC.

[Mark Sapiro (msapiro)]

I've been thinking about this report and patch, but hadn't formed a clear idea of a response. Now comments #2 has arrived which is really a separate issue.

With respect to comment #2, it asserts "when the list suppresses sending to a given user, it rewrites mail headers to drop that user from cc". This is backwards. A member's address is removed from Cc only if the member's "avoid duplicates" setting is off, not on. This only affects the scenarios in that the premise becomes "User A is a user with the feature disabled".

With respect to the original proposal/patch, I'm concerned that the patch only bypasses the Cc alteration when the From: domain publishes a relevant DMARC policy and won't be otherwise changed. It fails to consider whether other DKIM signature breaking transformations such as subject prefixing, addition of headers/footers and content filtering are being done. I'd be more inclined to accept a new "don't break DKIM" setting which if enabled would trump all settings that alter the message in a DKIM signature breaking way.

However, that all said, There are a few other considerations. I will soon release Mailman 2.1.30 which will be the last Mailman 2.1 release from the GNU Mailman project. I think the way to deal with DMARC in the future is with ARC, https://www.rfc-editor.org/info/rfc8617. There is ARC support in Mailman 3, although I think it can also be implemented in the MTA without involving Mailman.

Also, I have never fully understood why Cc dropping is dependent on the member's "avoid duplicates" setting at all (this predates my involvement with Mailman). It is intended to prevent Cc lists from growing without bound in long threads where everyone is doing reply-all, but why only Cc: and not To:, and why only addresses that don't "avoid duplicates". In short, I'm reluctant to make a change without a clear direction which I don't have.

[Mark Sapiro (msapiro)]

My comment #3 is incorrect and comment #2 is correct in that "when the list suppresses sending to a given user, it rewrites mail headers to drop that user from cc" is correct, not backwards as I claimed. I.e., the paragraph
```
With respect to comment #2, it asserts "when the list suppresses sending to a given user, it rewrites mail headers to drop that user from cc". This is backwards. A member's address is removed from Cc only if the member's "avoid duplicates" setting is off, not on. This only affects the scenarios in that the premise becomes "User A is a user with the feature disabled".
```
should be ignored and the final "why only addresses that don't "avoid duplicates" should be "why only addresses that "avoid duplicates".

My apologies for my confusion and any that I caused in others.

[Ian Kelling (iank)]

Mark, regarding this point "It fails to consider whether other DKIM signature breaking transformations such as subject prefixing, addition of headers/footers and content filtering are being done." It checks that no DMARC mitigation is on. In that case, having any DKIM signature breaking is already going to send invalid DMARC email, which seems to me to be a bigger separate problem: I don't think mailman should allow that combination of settings at all: don't send dmarc failing messages. Also, in the case the list has no DMARC mitigation and is breaking DKIM, not modifying the cc here will also prevent mailman from sending as many DMARC failing messages, because it will detect more duplicates and not send them. And, as you said, mailman 2 is going away. So, I think the patch is good as is.

[Mark Sapiro (msapiro)]

I've been trying to decide what to do about this, and I've come up with a plan. My issue with the @iank approach is it focuses solely on DMARC and I think there could be other valid reasons for not wanting to break DKIM in general regardless of DMARC. I considered a 'dont_break_dkim' switch that would trump any message modifications, possibly based on there being a DKIM sig in the message and what was signed, but even without those conditions this is too big a change this late in the MM 2.1 lifecycle.

So I've decided a 'drop_cc' switch with Yes the default and meaning current behavior, but No meaning don't modify the Cc: in AvoidDuplicates.py is just what @eblake is looking for and should also satisfy @iank as far as not breaking DKIM for DMARC is concerned even though it would apply to all messages and not just those where DMARC might be an issue.

[Ian Kelling (iank)]

Awesome. Thank you Mark! This makes the behavior consistent for all
users, which is probably the best thing overall. Note, this was
an issue in Mailman 3 too as far as I could tell from glancing at the
code.

[Mark Sapiro (msapiro)]

See https://gitlab.com/mailman/mailman/issues/647