Non-blocking DMARC mitigations should also be done for p=none

[Jakob Bohm]

If you read the DMARC informational RFC (no longer just a draft), you will see that there are 3 possible policies a mail domain (of a poster) can set

p=reject All recipients should reject/drop mails with From: ...<at ourdomain> that have neither DKIM not SPF pass for our ourdomain. Send automatic reports of this to our postmaster at the specified rua address.

p=quarantine All recipient spam filters should penalize ditto. Send the same automatic reports.

p=none Recipients should act as if this policy was not there, but send the automatic reports so we can decide if our policy needs fine tuning before switching to a tougher p value.

Currently Mailman 2.1.20+ has code that applies reasonable adjustments to avoid failures with posters from p=reject domains (such as Yahoo). Mailman 2.1.20+ also has an option to do the same for p=quarantine.

But when a domain is set to p=none because the postmaster is trying to figure out what will break if he/she/they turn on quarantine, Mailman will continue sending mail in a way which causes all the backscatter reports from everybody except the list owner.

As a postmaster I find this very annoying as it makes it hard to find actual problems in the flurry of backscatter noise from making even a few posts to a single Mailman list. Especially annoying is reports that some servers received mails with failed DKIM from a 4th party IP, as it is impossible to tell if those 4th parties were forwarding mails that were broken by Mailman munging the Subject header or were handling mails of some other kind needing a different correction.

I would therefore suggest the following change to the 2.1 branch (since 3.x is still not in a usable state):

* If dmarc_moderation_action is set to a value other than Reject or Discard, the specified mitigation should be applied to domains with any valid DMARC policy, not just reject.

The effect of this would be:

+ List admins need to do nothing extra, since this works with the existing Mailman settings.

+ Domains that use p=none with a user posting to a list with action other than Reject/Discard will see the true effects that setting p=reject would have, and their users will see any change in the Mailing list behavior and be able to complain to the local postmaster before the domain goes to p=reject. Which is exactly the intended purpose of p=none.

+ Domains that use p=none with a user posting to a list with action set to Reject/Discard will receive a flurry of error reports reflecting how badly things will break if they go to p=reject, but the users will not loose their ability to post (because the reject/discard action is not applied to domains with p=none). Again this is consistent with the intended purpose of p=none.

+ Domains that use p=quarantine with a user posting to a list that has not set dmarc_quarantine_moderation_action will see the same effects as domains that use p=none (see above).

+ Domains that use p=quarantine with a user posting to a list that has set dmarc_quarantine_moderation_action will see the same behavior as for the current 2.1.20 code.

+ Domains that use p=reject will see the same behavior as for the current 2.1.20 code.

[Mark Sapiro (msapiro)]

I think I see what you are saying. If I have it right, you admin a domain that publishes DMARC p=none to get reports to assess the effect of going to a stronger DMARC policy, but you see reports due to Mailman list mail because Mailman does not apply DMARC mitigations to mail From: domains that publish DMARC p=none. So you would like Mailman to apply disruptive message transformations to mail From: your domain that would be transformed if you applied a stronger DMARC policy.

My opinion is DMARC and mailing lists do not play well together. DMARC was originally intended to be used by domains such as those of financial institutions where mail From: those domains would only be for business purposes and not posted by individuals to mailing lists. It was mis-used by Yahoo and later AOL in a misguided attempt to make upo for prior security lapses, but was never intended to be used by domains that provided email service to users for personal use.

While I don't understand what your domain is or what it is used for, I suggest that if individuals are allowed/encouraged to send personal mail From: your domain to email lists, that DMARC p=quarantine or p=reject policies are inappropriate.

[Jakob Bohm (jb-debbugs)]

Please reconsider.

In my specific case, these are domains where regular users (including me) send mails. I am adding DMARC (initially in p=none mode) because recipients are beginning to refuse direct e-mails not having a DMARC policy, and that is obviously bad for business.

I am referring to the use of "p=none" as a way to test the waters, because that is the RFC defined purpose of that setting.

While you may be politically opposed to DMARC, it has now become a reality of how large parts of the e-mail system works, and so it can no longer be ignored or omitted outside very small gated communities of people not e-mailing to and from the users that are signed up to the big providers or to any other providers pushed into setting up DMARC for interoperability.

So ignoring DMARC is no longer a realistic option, and neither are the "Accept/Reject/Discard" Mailman settings that simply ignore the problem or penalize users for using domains that try to keep up with best current practices.

Hence my suggestion that DMARC handling be done in ways that maximize interoperability, rather as some obscure "protest against Yahoo! Inc" special case.

The only conflict between mail lists and DMARC (and ADSP) is that forwarding or generating mail with someone else's From header is no longer OK, just as SPF previously made it no longer OK to use someone else's envelope "MAIL FROM" domain.

[Mark Sapiro (msapiro)]

If I understand correctly, what you are saying is that by publishing DMARC p=none you receive reports of Mailman list mail that fails DMARC and some of these reports MIGHT be spurious because if your DMARC policy were quarantine or reject the lists MIGHT apply Mailman's DMARC mitigations.

I will consider addressing this, but not in the way you suggest. I will consider adding another list setting dmarc_none_moderation_action - Shall the above dmarc_moderation_action if Munge From or Wrap Message apply to messages From: domains with DMARC p=none as well as p=reject and p=quarantine? with the more detailed help explaining that a Yes setting only makes sense if dmarc_quarantine_moderation_action is also Yes and also explaining that the intend is to suppress the reports to the domain owner of failures that wouldn't be failures if the domain owner's policy were quarantine or reject.

I will not do this unconditionally because I believe it should be up to the individual list owner to decide whether she wants Munge From or Wrap Message applied to messages that should be accepted without these transformations.

[Stephen Turnbull (stephen-xemacs)]

I believe the OP can get exactly the effect he wants by specifying a DMARC policy record including "p=quarantine; pct=0". Mailman doesn't (and arguably shouldn't) check the value of pct, let alone roll dice, on p=quarantine or p=reject, so this policy will cause Mailman to mitigate (if mitigation is available in that version of Mailman and requested by the list admin). On the other hand, pct=0 means no conforming recipient implementation will actually quarantine any messages. (Yes, pct=0 is explicitly permitted by the RFC.) The vast majority of mail is covered by conforming implementations according to DMARC.org testing (but I don't know that this corner case was tested -- someone with a test domain can do that easily enough, and if someone does, please report whether it worked and which providers were tested to <email address hidden>).

[Jakob Bohm (jb-debbugs)]

Stephen, using a different domain setting to compensate for mailman misbehavior is the wrong way.

There is a DMARC RFC (technically Informational due to the bad state of the relevant IETF WGs, as seen in the SPF/SenderID fiasco). This explicitly states what proper e-mail software should and should not do when a domain declares "p=none".

Mailman is intentionally disregarding the RFC and causing its default behavior to cause problems for postmasters whose users are permitted to participate in mailing lists, when the listmasters of those lists have unfortunately chosen to use Mailman and follow whatever Mailman documentation says is the suggested way to configure it. Fortunately, some Mailman listmasters are waking up and choosing settings that are as close to compliance as Mailman allows.

I am asking for RFC compliance and interoperability, not workarounds, and especially not workarounds that need to be done by 3rd party postmasters and depend on listmasters making non-default settings (such as changing Mailman settings for domains with p=quarantine).

It is also worth noting that the restrictions imposed by DMARC (do not forwarding mails with unchanged From header unless some other rules are followed and applicable) is the unfortunate result of most common e-mail MUA programs (At least Thunderbird and Outlook) failing to tell recipients when the From: header differs from the MTA-checked envelope from address, a limitation which has been exploited by spammers and scammers for many many years with no MUA fix in sight.

[Mark Sapiro (msapiro)]

Jakob writes "Mailman is intentionally disregarding the RFC and causing its default behavior to cause problems for postmasters whose users are permitted to participate in mailing lists ..."

I assume the RFC you are referring to is RFC 7960. I have read this RFC and I do not draw the same conclusion. In particular, section 2, paragraph 2 says

   Note that domains that assert a "p=none" policy and email messages
   that pass standard DMARC validation do not have any interoperability
   issues.

which implies to me that it is not necessary to apply any mitigations to messages From: domains publishing p=none.

Clearly you believe otherwise. Please identify specific RFC sections and language that support your opinion.

The mitigations of modifying the From: header or wrapping the message have negative consequences for users of at least some MUAs, so list owners have incentives to apply them to as few messages as possible. We are currently working on the mitigation strategy for Mailman 3 and have decided at this point to not have controls over which policies trigger mitigation, but when mitigating based on policy to apply mitigation for p=reject and p=quarantine, and not p=none. If we are convinced that RFCs say we SHOULD or MUST apply munge/wrap mitigations to messages from p=none domains, we will do that, so please tell us why you think they do.

[Jakob Bohm (jb-debbugs)]

No, RFC 7489, specifically the configuration instructions in section B.2.1

RFC 7960 at first glance (haven't read it end to end) seems to be an attempt to document implementation status in mailing list software at the time of publication.

[Barry Warsaw (barry)]

B.2.1 are just part of the examples, but even there it says: "Receivers should not alter how they treat these messages because of this DMARC policy record ("p=none")". This reinforces that no mitigations should be applied to p=none.

[Mark Sapiro (msapiro)]

To elaborate on Barry's comment, RFC7849 defines DMARC and the policies and reporting requests that domain managers can publish to implement DMARC and the things that receivers of mail can do to implement the published policies and reporting requests of From: domains. In this context, Mailman is not a receiver, but rather is a mediator as defined in section 5.3 of RFC5598 and discussed in section 5.2 of RFC7960.

I understand Jakob's issue and I understand that the implementation of dmarc_none_moderation_action doesn't really solve that because it relies on the cooperation of multiple third party mailing list operators to enable it even though they have a disincentive to do so. It is precisely because of this that that implementation is not being carried forward to Mailman 3.

The underlying issue is that the original design of DMARC did not take into account the role of mediators in mail delivery. RFC7960 is an attempt to clarify the issues and suggest mitigations that mediators can apply now and in the future, but none of this requires or even suggests that a mediator should treat a p=none domain differently from a "no policy" domain for purposes of applying DMARC mitigations.

I firmly believe Jakob's issue is with DMARC and not with Mailman and it is not up to Mailman to fix it.

[Jakob Bohm (jb-debbugs)]

You are getting this all backwards.

While I agree that DMARC interacts badly with mailing lists in general, writing mailing list software in ways that make this worse is not going to help anybody.

DMARC as it exists today has a p=none feature which serves no other known purpose than allowing postmasters to detect which, if any, problems will occur for their particular domain, should they publish a stricter policy.

This detection mission is completely frustrated if mailing lists that are configured to *not* cause problems if DMARC is set to an active policy continue cause false alarms in the p=none test mode.

I firmly believe that the wording about not taking different action for "p=none" is directed only at recipients that would otherwise reject or penalize mail based on the DMARC policy, not at mail forwarders deciding if the e-mail should be handled in a DMARC-compatible way, e.g. by enabling workarounds.

[Mark Sapiro (msapiro)]

Clearly we disagree on this and I don't think either of us will convince the other to change.

Mailman is trying to enable list owners to cope with DMARC in a way that minimizes negative impacts on lists and their subscribers. This is an evolving landscape. Some Microsoft services already are treating mail with the same address in From: and To: as suspicious which impacts mail currently DMARC mitigated by Mailman (as well as mail from anonymous lists).

There are initiatives in progress to enable a mailing list to certify that outgoing mail passed DMARC when it was received by the list (Authenticated Received Chain <https://datatracker.ietf.org/doc/draft-ietf-dmarc-arc-protocol/> and <https://datatracker.ietf.org/doc/draft-ietf-dmarc-arc-usage/>). Thus, Mailman's handling of these things will likely change in the future.

However, Mailman 2.1's DMARC mitigations are not likely to change, and for the near term at least Mailman 3 is not going to apply mitigatations to p=none mail unless the list owner chooses to apply mitigations to all mail (similar to from_is_list in MM 2.1). Mailman developers believe this is the correct approach and is RFC compliant.