Make Launchpad DMARC Compliant to avoid Launchpad mail considered spam

Bug #1589693 reported by Richard on 2016-06-06
102
This bug affects 19 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Undecided
Unassigned

Bug Description

ISSUE:
When a comment is made on a bug, that comment is sent out by Canonical servers using the poster's email address. This won't work for domains that have dns policy records for email such as spf dkim or dmarc.

IMPACT:
A. ALL notifications where the issuer's domain has a published policy will be treated as spam or rejected
B. Canoncial's outbound smtp ip will be spam rated
C. dmarc policy holders will get reports on recipient's spammed domains
D. It is a bit of a privacy issue to distributes recipients domain names.

This won't work in 2016.

RECOMMENDED FIX:
Either use some clever canonical.com email construct like <email address hidden>
or a <email address hidden> address

Launchpad is completely compliant with domains that use SPF and DKIM, as the envelope sender is @canonical.com. It's only operators who unwisely use a restrictive DMARC policy for user domains (breaking mailing lists, among other things) that are a problem.

summary: - Launchpad mail is spam for dkim/spf/dmarc policy domains
+ Launchpad mail is spam for user domains with restrictive DMARC policies
Richard (ismail-a) wrote :

Here are the dmarc recommended options:

https://dmarc.org/wiki/FAQ#I_operate_a_mailing_list_and_I_want_to_interoperate_with_DMARC.2C_what_should_I_do.3F

Any "good" option will require modifying the from field, signing using dkim, supporting spf and provide a publicly verifiable certificate when sending.

It seems canonical does not do any of the four.

Richard (ismail-a) on 2016-06-09
summary: - Launchpad mail is spam for user domains with restrictive DMARC policies
+ Make Launchpad DMARC Compliant to avoid Launchpad mail considered spam
Richard (ismail-a) wrote :

current mail headers:
Reply-To: Bug 1589693 <email address hidden>
Received: from indium.canonical.com (indium.canonical.com [91.189.90.7])

Trouble 1: no SPF record
Trouble 2: no DKIM signature
Trouble 3: use of outdated TLS 1.0, should be 1.1 or 1.2
Trouble 4: MTA does not present a globally trusted X.509 certificate for its dns name

Felix Schwarz (felix-schwarz) wrote :

William Grant (wgrant) wrote on 2016-06-06:
> It's only operators who unwisely use a restrictive DMARC policy for user domains
> (breaking mailing lists, among other things) that are a problem.

Well, I think that ship has sailed. Big providers are employing strict DMARC policy and much more providers are willing to enforce these.

IMHO we just have to accept that email is not "anyone can send anything" anymore.

I just found a few launchpad messages in my spam folder. Besides Canonical having a DMARC record but no DKIM signature/SPF record the main culprit seems to be the From header which contains a commenter's email address.

Well that goes directly against DMARC and I can somehow understand their reasoning even if it is a bit annoying at times. Still I hope launchpad could be reconfigured to use an email address "@canonical.com" or "@bugs.launchpad.net".

Thomas Ward (teward) wrote :

Forgive me for a very blunt statement, but in reply to wgrant's statement about LP being compliant, it actually *isn't* being compliant. Simply altering "Sender" field and then implementing no changes to the "From" field is going to trigger any and all DMARC rules that have either a QUARANTINE or REJECT policy in place.

In having to handle DMARC compliance on a listserv implementation, and doing some heavier-duty testing with my DMARC compliant mail server(s), I discovered that not only does the Sender address and the outer Envelope have to not be from the DMARC-compliant addresses, the From field in headers is *also* verified, and can (and usually will) result in a failure case for all DMARC policies which restrict mail.

A large number of companies and groups implement strict DMARC handling, of which Launchpad will be noncompliant for in the future.

A prime example of this is a message sent by Launchpad today to the user 'nacc' which addressed the email as from me, because of my reply to a merge proposal made by them on something, and my critique on how they could fix an issue in that proposal. The relevant header bits are here:

Authentication-Results: mx.google.com;
       spf=pass (google.com: best guess record for domain of bouncesATcanonical.com designates 91.189.90.7 as permitted sender) smtp.mailfrom=bouncesATcanonical.com;
       dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=QUARANTINE) header.from=trekweb.org
...
From: Thomas Ward <email address hidden>
Sender: bouncesATcanonical.com

DMARC only cares about SPF and DKIM; it does not specifically care about whether messages were handled with TLS or not. Specifically, it wants the SPF of the sender and the DKIM of the "From" address to both validate properly. SPF passes because the Sender SPF records match properly with Canonical's SPF records. DMARC however fails because the domain trekweb.org publishes a DKIM record, and the DKIM signature does not match.

In *most* ListServ implementations that are DMARC compliant, there sits a mechanism to replace the "From" address with the address of the ListServ itself (sometimes with the raw mail header of "From: John Smith <listserv_AT_domain.com>", and sometimes a Reply-To of either the actual originator. The "From" field, therefore, validates against the domain of the listserv / system itself and *not* the actual originator. This solves the DMARC Compliance issue, as DMARC then checks the domain of the system itself (in this case, it would be canonical.com) and that will not fail.

Perhaps a setting could be added to user settings that would be "mask email address in notifications", which would then under the hood set the From header to be something like "User Name <bouncesATcanonical.com>" instead of the original from address?

Starbeamrainbowlabs (sbrl) wrote :

Just stumbled across this today. The core of the issue is that SPF and DKIM restrict who can send an email in the name of an email server - as I understand it DMARC is primarily a reporting system. To that end, it's not just DMARC that's the issue - SPF and DKIM are already causing issues too.

An email server's reputation is absolutely key to being able to deliver mail, and to that end most providers will implement fairly strict SPF and DKIM policies (to say nothing of DMARC) in order to protect their reputation.

I'd recommend rewriting to something like <email address hidden> or something.

Jonathan Kamens (jik) wrote :

I just started running opendmarc on my mail server, and it informed me this evening that an email message received from LaunchPad failed DMARC. The message came from `<email address hidden>`, so it's Canonical's own domain that is failing DMARC in this case, because although the domain has a DMARC record, it doesn't have an SPF record and launchPad doesn't do DKIM signatures.

Regarding the other issue discussed above, i.e., LaunchPad putting commenters' email addresses in the From: line of the message, this is clearly wrong given the status quo of email security. While I suppose someone could have reasonably claimed otherwise two and a half years ago, that ship has sailed. You can't be generating email messages that violate other domains' DMARC policies, period.

Both of these issues really need to be fixed.

On 28/09/18 09:24, Jonathan Kamens wrote:
> I just started running opendmarc on my mail server, and it informed me
> this evening that an email message received from LaunchPad failed DMARC.
> The message came from `<email address hidden>`, so it's
> Canonical's own domain that is failing DMARC in this case, because
> although the domain has a DMARC record, it doesn't have an SPF record
> and launchPad doesn't do DKIM signatures.

Your DMARC configuration seems to be buggy, since the DMARC record for
canonical.com says no action should be taken -- it's just for testing.

_dmarc.canonical.com. 600 IN TXT "v=DMARC1; p=none; pct=100;
rua=mailto:<email address hidden>"

Jonathan Kamens (jik) wrote :

>Your DMARC configuration seems to be buggy, since the DMARC record for canonical.com says no action should be taken -- it's just for testing.

My DMARC configuration is not buggy. I didn't say that my mail server rejected the message, I said that it notified me that the message failed DMARC, which it did. The fact that the domain's DMARC policy says that no action should be taken about failing messages is orthogonal to the fact that the message in question failed DMARC.

I'm commenting just now about this because I apparently never got the notification about the comment above. I assume that's because it was dropped due to DMARC, although I can't say for sure since I only have email logs back to October 7.

However, I can say for certain that I'm missing other notifications from launchpad due to DMARC. My logs show that a few days ago I Launchpad emailed me about a comment added to a bug by a user with a yahoo.it email address. Launchpad put his email address in the From: line as outlined here in this bug, and yahoo.it has a p=reject DMARC policy, so my mail server rejected it.

Not, cool, Canonical. Please fix this.

Starbeamrainbowlabs (sbrl) wrote :

This is akin to impersonation. To that end, there's absolutely no reason for this to continue. Is Launchpad's code open-source?

Colin Watson (cjwatson) wrote :

The messages in question represent actions taken by the user whom Launchpad then lists in the From: line, and in most cases consist principally of body text that that user wrote. This is only "impersonation" if you take a very narrow MTA-centric view of who the person is. Morally, the situation is more like that of mailing lists resending messages they receive from users (with the main differences being that the origin of the message might be a mail user agent or might be an HTTP client, and that in some situations Launchpad will generate body text representing the user's actions rather than solely passing on lightly-modified body text provided by the user). If you're subscribed to this bug and you receive this message from me, it really is in essence a message from me and not from Launchpad.

Now, I'm not in general opposed to changes that adjust Launchpad's email headers so that it doesn't trip over DMARC implementations, although it would need quite a bit of care to go through all the relevant call sites and make sure that any such changes preserve the proper natural reply behaviour, and we should consider things like whether we want a From: address per (e.g.) bug or per Launchpad user. All this would take a fair amount of effort and discussion.

We may indeed nowadays be in an environment where we have little choice; but let's be clear that it would be a change made for operational reasons to improve the reliability of our mail delivery, and not for essentially moral reasons such as impersonation.

As for Launchpad being open source, yes it is, and there's a link with details at the bottom of every page on launchpad.net.

Jonathan Kamens (jik) wrote :

>We may indeed nowadays be in an environment where we have little choice;

I don't think there is any doubt about that, and anyone who continues to express doubt is either ignorant or naive about the current state of email.

I just missed another notification from Launchpad this morning because the comment I was being notified about was posted by a user at a domain with a DMARC reject policy.

Some notifications from Launchpad are not being delivered successfully because of how Launchpad is sending them. This isn't going to change; in fact, it's going to get worse over time. Plugging one's ears and loudly singing "la la la I'm not listening" is not going to change these facts.

Frankly, I can hate on DMARC as much as anybody, but I don't think pissing into the wind is a good long-term strategy.

>but let's be clear that it would be a change made for operational reasons to improve the reliability of our mail delivery, and not for essentially moral reasons such as impersonation.

100% agreed.

Andrew Johnson (anj) wrote :

A colleague who uses gmail for his MUA has been complaining for some time that he never gets Launchpad notification emails for comments that I make on his bug reports and merge requests, but he does get them from other team members. It's now clear to me having found this bug report that this Launchpad behavior is the reason for that problem. My employer publishes a strict DMARC policy (p=reject), whereas the other people he interacts with through Launchpad have either a lenient DMARC policy (p=none) or don't configure DMARC at all.

Andrew Johnson (anj) wrote :

I just found a similar and much older bug #31586 which doesn't mention DMARC until comment 33, but it is describing the same problem. This bug might therefore be a duplicate of that one, but this one has more specific details about DMARC which should be raising the urgency of a fix IMHO.

Scott Kitterman (kitterman) wrote :

There is an upcoming bolt on protocol for DMARC, called ARC:

https://tools.ietf.org/html/draft-ietf-dmarc-arc-protocol-23

This is the best shot at solving the problem with DMARC and mailing lists and as I understand it, the major mail providers have implemented or are planning to implement it. It's post IETF last call and in the RFC editor queue for publication.

Up to now, I think a reasonable case for claiming similarity between LP and a mailing list could be made. In an ARC world though I think that's less the case because the preferred mode of operations requires an inbound DKIM signature which, by definition, LP can't provide.

Operationally, this is only going to to get harder the longer a change isn't made. It' probably not relevant, but dkimpy (which last I knew LP used for DKIM verification) also supports ARC in recent versions.

Scott Kitterman (kitterman) wrote :

ARC is probably relevant for LP mailing lists as it gives a way to avoid from munging.

Jonathan Kamens (jik) wrote :

This bug isn't about mailing list hosted on Launchpad, it's about notifications generated by Launchpad. ARC has nothing to do with solving that problem.

Andy Brody (abrody) wrote :

Launchpad is really completely in the wrong here as far as DMARC compliance for its notification email is concerned.

The same rules apply to Facebook, GitHub, and any other site on the Internet. If the user's domain doesn't list you as an authorized sender, you shouldn't be impersonating them with the "From:" address.

Launchpad isn't somehow special here. Launchpad's email notices are indistinguishable from malicious forgery.

I was also very unpleasantly surprised that in addition to violating my domain's DMARC policy, Launchpad is disclosing my private email address despite my having checked the "Hide my email addresses from other Launchpad users" option.

Jonathan Kamens (jik) wrote :

I just missed an important notification from Launchpad about a bug I filed because the person who made the comment which generated the notification has an email address in a domain which enforces DMARC.

Jonathan Kamens (jik) wrote :

I just missed another notification because the person who made the comment which generated the notification has an email address in a domain which enforces DMARC.

It's unbelievable that this still isn't fixed over three years after the maintainers of Launchpad became aware of the issue.

Colin Watson (cjwatson) wrote :

We've been very short-staffed for a while, but are in the process of
growing our team again. This is too big a job to fit into spare cycles,
so I've nominated this for our roadmap for the next cycle.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Related questions