Launchpad should batch attachment notification emails

Bug #424849 reported by Marco Rodrigues
106
This bug affects 15 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Graham Binns
apport (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

Binary package hint: apport

Hi!

The apport-collect currently add one attachment per comment, and it's a bad idea. Why not just add one comments with all the attachments inside like the apport-gtk does ?

Thanks

---

Proposed solution from one of the duplicate bugs:

* If there are a lot of updates in a short period of time, only send 1 email

Revision history for this message
Martin Pitt (pitti) wrote :

I'm just using launchpadlib.addAttachment(); unfortunately there is nothing that I can do about bug mail on the launchpadlib client side. But I agree that Launchpad bug mail for followup comments/attachments should be condensed into one mail, similar to what happens with initial bug filing.

Changed in apport (Ubuntu):
status: New → Invalid
Changed in malone:
status: New → Confirmed
summary: - apport-collect: don't add one attachment per comment
+ merge multiple attachment emails into one
Revision history for this message
komputes (komputes) wrote : Re: merge multiple attachment emails into one

I find that Launchpad should wait until all attachments from apport have been sent in before sending an email notification (notifying about all attachments). Currently (in some cases) Launchpad sends one email per apport attachment creating a lot of pollution in the users email inbox.

Revision history for this message
Gabe Gorelick (gabegorelick) wrote :

This really makes apport-collect almost unusable for me (apport-collect on a linux bug generate 26 (!) separate comments/attachments, most of which don't get threaded together in gmail). The real solution is obviously for Launchpad to just allow multiple attachments per comment, which solves the problem of multiple attachments also cluttering the bug report. Until that is implemented (see bug 82652), a good solution would be for Launchpad to just merge the emails notifying you of an attachment right after another attachment. This is already done when a comment follows right after a bug status change, for example.

Revision history for this message
Graham Binns (gmb) wrote :

The problem is simply that Launchpad treats each new comment as a boundary by which to split up notification emails (so if you make several changes to a bug and add a comment the changes and the comment will all arrive in the same notification, whereas if you comment first and then make the changes you'll get separate emails).

Now, from the difficulty-of-implementation point of view I think it's actually simpler (and more sensible) to support the multiple-attachments-per-comment use case via addAttachment() (and eventually via the UI) than it is to change the way Launchpad handles the notification boundaries. That said, we bundle up notifications for comments imported from remote bugs, though I can't remember off the top of my head how that works. It may be that we can re-use that.

I'm marking this as Triaged, Low on the basis that we should probably add the multiple upload functionality rather than fix this bug.

Changed in malone:
status: Confirmed → Triaged
importance: Undecided → Low
Leo (leorolla)
description: updated
Revision history for this message
Leo (leorolla) wrote :

Indeed they are not threaded in gmail...

e-mails are already condensated when you first file the bug.
e-mails are already condensated when you commend and change the status right away.
Why not going radical and waiting until a 5 minutes are gone?

Whatever a user does waits in a sandbox and after a lapse of time the changes become permanent (like in the wiki pages) and after that just one neat e-mail is sent. Bugs are not evolving that fast so that we couldn't possibly wait for 5 minutes.

Revision history for this message
Josh Brown (joshbrown) wrote :

Having a set time to wait before changes are applied seems like a bad idea. Perhaps we could merge the status changer with the add comment box, so that no changes are made until the submit button is pressed.

Revision history for this message
Leo (leorolla) wrote :

At least set a time to wait until mails are fired?

Revision history for this message
Leo (leorolla) wrote :

Usually there are many satus changes... there can be changes in the title and description as well... together with a comment and even a postscriptum... then add 28 attachements to it. What's the point of sending more than 1 e-mail ?

Revision history for this message
Josh Brown (joshbrown) wrote : Re: merge multiple notification emails into one

I think this bug needs a more ambiguous title considering the duplicate bugs.

summary: - merge multiple attachment emails into one
+ merge multiple notification emails into one
Graham Binns (gmb)
tags: added: story-better-bug-notification
Graham Binns (gmb)
summary: - merge multiple notification emails into one
+ Launchpad should batch attachment notification emails
Changed in malone:
importance: Low → High
Revision history for this message
Josh Brown (joshbrown) wrote :

Graham, this bug concerns more than just attachment emails - it is also about changing tags, summary, importance, etc.

summary: - Launchpad should batch attachment notification emails
+ Launchpad should batch notification emails
Revision history for this message
Graham Binns (gmb) wrote : Re: Launchpad should batch notification emails

Josh, the original report was about the problem with notifications generated when apport uploads attachments to Launchpad (the original summary was ambiguous; I'm looking at the description here). That's specifically about attachment notifications.

We're about to start work on a big push to fix a lot of bugs in subscriptions and notifications code. We'll use this bug to track the fact that attachment notifications should be batched. We will doubtless look at batching further, but fixing this specific bug is a nice first step.

summary: - Launchpad should batch notification emails
+ Launchpad should batch attachment notification emails
Revision history for this message
Josh Brown (joshbrown) wrote :

Graham: In that case, should I report a new bug with a more ambiguous summary?

Revision history for this message
Leo (leorolla) wrote :

Is it simpler to detect and batch attachments than just replacing on-the-fly notifications by 5-minute-interval digests?

Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 424849] Re: Launchpad should batch attachment notification emails

On 4 June 2010 20:10, Leo <email address hidden> wrote:
> Is it simpler to detect and batch attachments than just replacing on-
> the-fly notifications by 5-minute-interval digests?
>

At this point, yes.

The bug notification code (which can be found in
lib/canonical/launchpad/mailnotification.py [1] for anyone who wants
to take a look) is pretty ugly and is in sore need of refactoring. As
part of that work we may look at how batching works and whether we
need to do something different, but that's not the first thing on our
list of priorities. This bug is causing a lot of noise, so it's better
to fix this sooner rather than later.

 [1] http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/files/head:/lib/canonical/launchpad/mailnotification.py

Revision history for this message
Deryck Hodge (deryck) wrote :

Don't we batch in 5 minute (or some such) intervals now? Except for comments and attachments, of course.

I thought that was the point of this bug, to enable batching for attachments.

Revision history for this message
Leo (leorolla) wrote :

Well, when there are many consecutive comments I have the impression that I
get many consecutive e-mails? Is this wrong impression?

The idea is that many comments sent in the same block of time don't result
in many e-mails.

I rarely see something batched. The most common is a status change following
or followed by a comment, which result in only one e-mail for both actions.

Revision history for this message
Graham Binns (gmb) wrote :

On 6 June 2010 01:46, Deryck Hodge <email address hidden> wrote:
> Don't we batch in 5 minute (or some such) intervals now?  Except for
> comments and attachments, of course.
>
> I thought that was the point of this bug, to enable batching for
> attachments.

We batch in five minute intervals but comments mark the boundaries
between batches, so if you make a bunch of changes and add a comment
and then someone else makes a bunch of changes you'll get two separate
notifications. Attachments cause problems because they each get their
own comment, so we send out one notification per attachment. Otherwise
our batching is quite sane at the moment.

We *might* be able to move to five-minute digests when we've fixed all
the Launchpad-fakes-email-addresses bugs. The easy fix to some of
those bugs is to start to use <$<email address hidden>> as the From
address for notifications and if we do that we may be able to change
notification emails to look like this:

 From: Bug $FOO <$<email address hidden>>
 ...
 ** Graham Binns (gmb) made the following changes
   * In Ubuntu:
     * status Incomplete -> Confirmed
   * Comment added:
     This new digest thing is the bomb isn't it?

 ** Deryck Hodge (deryck) made the following changes:
   * In Malone
     * status Incomplete -> Fix Released
   * Comment added:
     Yeah, it's not bad.

But I'm not yet convinced that there would be a massive reduction in
bug mail traffic if we did that, or at least not a reduction directly
related to the batching of attachment notifications.

Revision history for this message
Deryck Hodge (deryck) wrote :

Thanks for the detailed explanation, Graham. That meshes with my understanding of what we do, too. That is a nice example of the advantage to sending mail from the bug rather than the person, but I think it should be a separate concern and we should consider this bug fixed when we batch attachments from the same person. That's an easy fix and and easy win.

We should open a bug about batching comments themselves, assuming we can find a nice way to do that. I would have thought that bug existed, but I cannot find one. Otherwise, as Graham says, we do this genrally okay now. It's just not obvious to people because comments are common, frequent, and can break batching of other changes in unexpected ways.

Cheers,
deryck

Graham Binns (gmb)
Changed in malone:
assignee: nobody → Graham Binns (gmb)
status: Triaged → In Progress
Changed in malone:
status: In Progress → Triaged
assignee: Graham Binns (gmb) → nobody
Graham Binns (gmb)
Changed in malone:
status: Triaged → In Progress
assignee: nobody → Graham Binns (gmb)
milestone: none → 10.06
Graham Binns (gmb)
Changed in malone:
milestone: 10.07 → 10.08
Graham Binns (gmb)
tags: added: story-better-notification-sending
Revision history for this message
Graham Binns (gmb) wrote :

Marking this as Triaged again since we won't be coming back to it until after the new subscription UI work is done.

Changed in malone:
status: In Progress → Triaged
milestone: 10.08 → none
Graham Binns (gmb)
Changed in launchpad:
status: Triaged → In Progress
Revision history for this message
Gary Poster (gary) wrote :

Graham investigated the status of aggregating attachments in notification emails.

Launchpad already aggregates attachments that are from the same person and for the same bug. Therefore, this bug, as described in its title and in the goals set out by Deryck and Graham, is fixed. I've marked it fix released.

However, we only allow each notification to contain a single comment. Apport includes comments with each attachment. Therefore, we might be able to solve the original complaint that led this bug to be filed by allowing notifications to aggregate comments (in additions to metadata and attachments).

I filed this as bug 768997.

Changed in launchpad:
status: In Progress → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

Gary does this mean that if apport were not to actually comment and only add attachments this would reduce the quantity of emails?

For example in https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-intel/+bug/750259/comments/7 - we can see apport adds a message of 'apport information'. If we were to remove that from apport would all the attachments being added be sent in one email?

Revision history for this message
Gary Poster (gary) wrote :

@Brian: yes, that's exactly what Graham reported.

Revision history for this message
Brian Murray (brian-murray) wrote :

Well the api requires a comment when adding an attachment so it doesn't seem that there is a way to modify apport to prevent multiple email messages.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 424849] Re: Launchpad should batch attachment notification emails

What if you have an empty comment, or does it reject that? If it
rejects it its probably a bug.

Revision history for this message
Graham Binns (gmb) wrote :

On 27 April 2011 19:09, Brian Murray <email address hidden> wrote:
> Well the api requires a comment when adding an attachment so it doesn't
> seem that there is a way to modify apport to prevent multiple email
> messages.

Ah, see, here's the problem:

Internally `comment` can be either a string or an IMessage, which is
how we can theoretically have several attachments linked to one
comment. In the declaration for IBug, we see:

    @operation_parameters(
        data=Bytes(constraint=attachment_size_constraint),
        comment=Text(), filename=TextLine(), is_patch=Bool(),
        content_type=TextLine(), description=Text())
    @export_factory_operation(IBugAttachment, [])
    def addAttachment(owner, data, comment, filename, is_patch=False,
                      content_type=None, description=None):

So, the problem is that the API won't let you pass a message in to
addAttachment(). If we want to change that we should probably track it
elsewhere.

Revision history for this message
Robert Collins (lifeless) wrote :

Wouldn't hangling '' as the commentin the same way the 'add
attachment' web form does be sane? Then the web form could be
ajaxified too :)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers