Email attachments instead of links to attachments

[Paul Everitt]

Backlog ID: 19 (Est. 12 hours)

Users in low-bandwidth situations might want an option to have "attachments" to content that generates alerts, actually attached to the email. That is, email is a more reliable service delivery for them than web browsing.

Thus, in cases where our email templates make URL links to files, we would also do file attachments to the email. The text of the email will look exactly the same, with the links. But some/all of these files will also be attached.

This option is not the default. It is something each user turns on. Unlike alert preferences (digest, off, etc.), it is not on a per-community basis. In fact, we aren't really saying an option of "Attach emails." Instead, the option is more like a section at the top of the Manage Communities screen:

     Optimize for low-bandwidth connections
     =====================================

     If your connection is intermittent or unreliable, KARL can optimize for this. For
     example, we will attach files to all email alerts (up to a limit), rather than provide
     links to download the files.

     Set for low-bandwidth? [] Yes [x] No

Make sure to system_name any mention of KARL.

Questions for OSI
==================

1) Do we keep the existing format, with the URL link, while also attaching the files to the email? Or do we do email attachments and get rid of the URL link? (Our preference is the former, in case you say "yes" to the next item.)

2) Do we put any upper limit on the size of individual (or aggregate) email attachments? If so, is this simply done as a system setting done by hosting admininstrators? In this case, some emails might be attached and some might not.

3) What is the exact text and UI for the changes to the "Manage Communities" screen?

[Paul Everitt (paul-agendaless)]

Chris, can you read up on this and look for any other questions I should ask?

[Chris Rossi (chris-archimedeanco)]

How should this interact with alert digests? Do you we really want to mail bomb users with an entire day's worth of attachments if there is a busy day for attachment uploads?

[Chris Rossi (chris-archimedeanco)]

As far as I can tell, there is no way for me to create an email attachment without loading the entire file into memory. This means, to avoid catastrophe, there must be a reasonable limit on the size of attachments as a reasonably small fraction of available memory. I would propose 1MB, but am open to other suggestions.

[Chris Rossi (chris-archimedeanco)]

Deployed to staging for testing.

[Chris Rossi (chris-archimedeanco)]

Just realized I haven't tested yet with digest. Will leave this open until I get a chance to do so. Have tested with non-digest alerts and it does work.

[Paul Everitt (paul-agendaless)]

I think we already face a catastrophe on huge files, as we allow them
to be downloaded over HTTP. Or is that mechanism a stream?

If it is a stream, and mail is truly different, let's up the limit to
5 Mb.

--Paul

On Sep 2, 2009, at 4:00 PM, Chris Rossi wrote:

> As far as I can tell, there is no way for me to create an email
> attachment without loading the entire file into memory. This means,
> to
> avoid catastrophe, there must be a reasonable limit on the size of
> attachments as a reasonably small fraction of available memory. I
> would
> propose 1MB, but am open to other suggestions.
>
> --
> Email attachments instead of links to attachments
> https://bugs.launchpad.net/bugs/422586
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in KARL3: In Progress
>
> Bug description:
>
> Backlog ID: 19 (Est. 12 hours)
>
> Users in low-bandwidth situations might want an option to have
> "attachments" to content that generates alerts, actually attached to
> the email. That is, email is a more reliable service delivery for
> them than web browsing.
>
> Thus, in cases where our email templates make URL links to files, we
> would also do file attachments to the email. The text of the email
> will look exactly the same, with the links. But some/all of these
> files will also be attached.
>
> This option is not the default. It is something each user turns
> on. Unlike alert preferences (digest, off, etc.), it is not on a
> per-community basis. In fact, we aren't really saying an option of
> "Attach emails." Instead, the option is more like a section at the
> top of the Manage Communities screen:
>
> Optimize for low-bandwidth connections
> =====================================
>
> If your connection is intermittent or unreliable, KARL can
> optimize for this. For
> example, we will attach files to all email alerts (up to a
> limit), rather than provide
> links to download the files.
>
> Set for low-bandwidth? [] Yes [x] No
>
> Make sure to system_name any mention of KARL.
>
> Questions for OSI
> ==================
>
> 1) Do we keep the existing format, with the URL link, while also
> attaching the files to the email? Or do we do email attachments and
> get rid of the URL link? (Our preference is the former, in case you
> say "yes" to the next item.)
>
> 2) Do we put any upper limit on the size of individual (or
> aggregate) email attachments? If so, is this simply done as a
> system setting done by hosting admininstrators? In this case, some
> emails might be attached and some might not.
>
> 3) What is the exact text and UI for the changes to the "Manage
> Communities" screen?

[Chris Rossi (chris-archimedeanco)]

File downloads are streamed. I see no way to stream email using existing
email packages. Remember that the limit is per attachment--there can be
more than one attachment per email, especially in digest mode. I suppose we
could try to have some more complex logic that looks at the potential total
size of the generated email.

Some numbers from production:

There are 21,084 blobs (most of which of are "files" on the site). Of
these, 1,702 are 1MB or greater, and of those 208 are 5MB or greater.

5MB probably won't kill us.

Chris

[Paul Everitt (paul-agendaless)]

I didn't claim email could be streamed. Quite the opposite. I noted
that email couldn't be streamed.

Let's go with 5 Mb.

--Paul

On Sep 3, 2009, at 11:48 AM, Chris Rossi wrote:

> File downloads are streamed. I see no way to stream email using
> existing
> email packages. Remember that the limit is per attachment--there
> can be
> more than one attachment per email, especially in digest mode. I
> suppose we
> could try to have some more complex logic that looks at the
> potential total
> size of the generated email.
>
> Some numbers from production:
>
> There are 21,084 blobs (most of which of are "files" on the site). Of
> these, 1,702 are 1MB or greater, and of those 208 are 5MB or greater.
>
> 5MB probably won't kill us.
>
> Chris
>
> --
> Email attachments instead of links to attachments
> https://bugs.launchpad.net/bugs/422586
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in KARL3: In Progress
>
> Bug description:
>
> Backlog ID: 19 (Est. 12 hours)
>
> Users in low-bandwidth situations might want an option to have
> "attachments" to content that generates alerts, actually attached to
> the email. That is, email is a more reliable service delivery for
> them than web browsing.
>
> Thus, in cases where our email templates make URL links to files, we
> would also do file attachments to the email. The text of the email
> will look exactly the same, with the links. But some/all of these
> files will also be attached.
>
> This option is not the default. It is something each user turns
> on. Unlike alert preferences (digest, off, etc.), it is not on a
> per-community basis. In fact, we aren't really saying an option of
> "Attach emails." Instead, the option is more like a section at the
> top of the Manage Communities screen:
>
> Optimize for low-bandwidth connections
> =====================================
>
> If your connection is intermittent or unreliable, KARL can
> optimize for this. For
> example, we will attach files to all email alerts (up to a
> limit), rather than provide
> links to download the files.
>
> Set for low-bandwidth? [] Yes [x] No
>
> Make sure to system_name any mention of KARL.
>
> Questions for OSI
> ==================
>
> 1) Do we keep the existing format, with the URL link, while also
> attaching the files to the email? Or do we do email attachments and
> get rid of the URL link? (Our preference is the former, in case you
> say "yes" to the next item.)
>
> 2) Do we put any upper limit on the size of individual (or
> aggregate) email attachments? If so, is this simply done as a
> system setting done by hosting admininstrators? In this case, some
> emails might be attached and some might not.
>
> 3) What is the exact text and UI for the changes to the "Manage
> Communities" screen?

[Chris Rossi (chris-archimedeanco)]

I didn't say you had. ;) I was just contrasting with file downloads, which you brought up. 5MB is fine.

Chris

[Chris Rossi (chris-archimedeanco)]

Note new wording from OSI:

Email Alert Attachments
------------------------------------------------------------------------------------------------------------------------------------------------------

Would you like to attach files directly to all email alerts? The default setting provides links to attachments in KARL. Please keep in mind that your selection will apply to ALL your KARL communities.

[x] No (Default, recommended)

[ ] Yes

[Chris Rossi (chris-archimedeanco)]

The digests took a little more work. All is checked in now and staging is updated.