Parent Task: More visiblity on distribution list processing

[Paul Everitt]

Note: This might be an uber-ticket with some smaller tickets as "children".

Dec 7 at 14:56 a user sent an email to KARL “distribution lists”. OSF uses KARL like Mailman, and has some reports in the People Directory which will expand and deliver mail. 24 emails were sent in, but they bounced before they could be redistributed.

At the time, I sent an email to Carlos/Nat with username details etc. I won't repeat here. Nat then replied with an explanation (and screenshots) about #Network Release and how it works.

Activities
==========

- Make a release of whatever is on master, just to make sure we are comfortable with making releases

- Make sure we have a good environment on staging (with something feeding it, e.g. test GSA) to experiment (this will be hard)

- Create a new UI for analyzing email activities throughout the chain of processing

- If someone sends an email to a bunch of distribution lists, does any throttling kick in?

- Make it easy to inspect the groups of the matched sender (e.g. it worked fine for Laura from GSA but not fine from a different user that didn't get KarlStaff from GSA) as well as any expired-password issue

- Greatly decrease the amount of logging mailin/mailout does (de-duplicate, send more to "debug" level instead of info)

- Greatly increase the amount of logging data that our web page shows from redis

- Consider an alternate mailin/out logging infrastructure, based in SQL, that allows filtering and drill down, in all parts of the system, from inbound queue -> postoffice -> mailin -> mailout -> /etc/mail.log

[Paul Everitt (paul-agendaless)]

Here's a ticket for today's 1PM discussion. I suspect we'll make individual tickets. The last bullet, for example, is a whopper.

[Paul Everitt (paul-agendaless)]

My notes from the call just now:

- Email bounce message saying password expired

- We need to recreate, on staging, the throttling concern for distribution lists (and even they are even throttled)

- Error messages (especially distribution list) should be meaningful and also should be logged and possibly alarmed

- The stderr type email messages that we send to /dev/null should go somewhere

- Move the var/mailout_stats_dir stuff into SQL

[Carlos de la Guardia (cguardia)]

I did the production release last week and it seems to have worked fine, so we are set with that preliminary.

After thinking about it for a while, I think it would be good for me to have individual tickets for this work. What do you think Paul?

[Paul Everitt (paul-agendaless)]

Done: https://bugs.launchpad.net/karl4/+bugs?field.tag=distlist <https://bugs.launchpad.net/karl4/+bugs?field.tag=distlist>

—Paul

> On Jan 14, 2018, at 10:40 PM, Carlos de la Guardia <email address hidden> wrote:
>
> I did the production release last week and it seems to have worked fine,
> so we are set with that preliminary.
>
> After thinking about it for a while, I think it would be good for me to
> have individual tickets for this work. What do you think Paul?
>
> ** Changed in: karl4
> Status: New => In Progress
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1741935
>
> Title:
> More visiblity on distribution list processing
>
> Status in KARL4:
> In Progress
>
> Bug description:
> Note: This might be an uber-ticket with some smaller tickets as
> "children".
>
> Dec 7 at 14:56 a user sent an email to KARL “distribution lists”. OSF
> uses KARL like Mailman, and has some reports in the People Directory
> which will expand and deliver mail. 24 emails were sent in, but they
> bounced before they could be redistributed.
>
> At the time, I sent an email to Carlos/Nat with username details etc.
> I won't repeat here. Nat then replied with an explanation (and
> screenshots) about #Network Release and how it works.
>
> Activities
> ==========
>
> - Make a release of whatever is on master, just to make sure we are
> comfortable with making releases
>
> - Make sure we have a good environment on staging (with something
> feeding it, e.g. test GSA) to experiment (this will be hard)
>
> - Create a new UI for analyzing email activities throughout the chain
> of processing
>
> - If someone sends an email to a bunch of distribution lists, does any
> throttling kick in?
>
> - Make it easy to inspect the groups of the matched sender (e.g. it
> worked fine for Laura from GSA but not fine from a different user that
> didn't get KarlStaff from GSA) as well as any expired-password issue
>
> - Greatly decrease the amount of logging mailin/mailout does (de-
> duplicate, send more to "debug" level instead of info)
>
> - Greatly increase the amount of logging data that our web page shows
> from redis
>
> - Consider an alternate mailin/out logging infrastructure, based in
> SQL, that allows filtering and drill down, in all parts of the system,
> from inbound queue -> postoffice -> mailin -> mailout -> /etc/mail.log
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/karl4/+bug/1741935/+subscriptions