KARL3

Bug #770428
Comment #8

Comment 8 for bug 770428

Revision history for this message

Paul Everitt (paul-agendaless) wrote on 2011-05-10: Re: [Bug 770428] Re: Develop email-in tracer tool

Maybe 4 hours to make the special-purpose tool, done in a way that makes sense. (We have some more things to discuss on this.) Probably 1 hour to make the error monitor report it, then an hour to get all parts of it into place. Add padding as appropriate.

--Paul

On May 10, 2011, at 11:59 AM, Nat Katin-Borland wrote:

> Good point about that much content potentially impacting search. About
> how much time would we need to implement something like this? Then we
> can run it by Tom. I don't want to wait too long to have something in
> place because we've already had 2 periods of down time in the last 6
> days and we're going to start to loose users if email isn't reliably
> delivered.
>
> -Nat
>
> --
> Nathaniel Katin-Borland
> Support Specialist
> Knowledge Management Initiative
> KARL Support Team
>
> Open Society Foundations - New York Office
> 400 West 59th Street
> New York, NY 10019
> Email: <email address hidden>
> Phone: 212-547-6984
> http://www.soros.org/
> http://www.karlproject.org
>
> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Chris Rossi
> Sent: Tuesday, May 10, 2011 11:18 AM
> To: Nathaniel Katin-Borland
> Subject: Re: [Bug 770428] Re: Develop email-in tracer tool
>
> On Tue, May 10, 2011 at 10:59 AM, Robert Marianski <
> <email address hidden>> wrote:
>
>> I was assuming these tracer emails would just be blog post entries,
>> and then the test would be to verify that unique text, like a uuid or
>> timestamp, was present on the page.
>>
>> What are the negative consequences of just sending an email to a
>> separate private community? If we want to eliminate the content feeds
>> side effect, we can create a separate dummy user just for this tracer
>> private community.
>>
>> I guess this makes the test to verify that the mail went through more
>> complicated, but it might be easier to move the complexity there than
>> to build something separate in karl.
>>
>>
> Well, actually, now that I remember, what we had actually discussed is a specialized blog tool in a specialized community. So, actually, all of the mailin machinery would be exercised but you wouldn't end up with an enormous number of blog entries stored unnecessarily in your database. If we're sending a tracer email every 5 minutes, that's 288 blog posts a day. After just a week, your test community has 2016 blog posts in it, and growing.
> That just doesn't seem sustainable.
>
> So the idea was to just create an object that looks for all the world
> like a blog for the mailin tool to write a blog entry to, but instead of
> storing each blog entry it would simply not the time it had last
> received a blog entry. So you still have a bunch of transactions, but
> at least not a ton of data you're trying to store unnecessarily. Zagy's
> idea also eliminates the unnecessary transactions, but by using this
> dummy blog tool, we still exercise all of the mailin machinery end to
> end.
>
> Chris
>
> --
> You received this bug notification because you are a direct subscriber of the bug.
> https://bugs.launchpad.net/bugs/770428
>
> Title:
> Develop email-in tracer tool
>
> Status in KARL3:
> New
>
> Bug description:
> We don't have a reliable way to know when email-in is wedged. Rather
> than trying to monitor each part of the end-to-end facility, we should
> instead monitor the failure of a tracer email to be received.
>
> The idea is that something, somewhere (hopefully inside the OSF NYC
> LAN) sends a tracer email to KARL every N minutes. This won't be a
> blog entry, as we don't want some dummy blog filling up and the
> content feeds getting blasted.
>
> Instead, we'll develop a new tool, either as part of a community or
> something hanging off the top. This tool will record receipt of the
> email. We'll also make sure that if the tracer wasn't received in the
> last N minutes, monitoring gets triggered.
>
> Some notes:
>
> - Exercise as much of the machinery as possible. We don't really need
> to test the ability to create a blog entry or blog comment. If it
> goes out of the NYC network, through the Internets tubes, into the
> outer Postfix, through its spam filters, to the inner, through
> repoze.postoffice, and is picked up by the daemon, that will handle
> 99% of the problems.
>
> - Make sure admins like Robert/Nat/Paul can go and see something about
> the tracer. Most likely the admin screen.
>
> - Document, in that tool, the email address that is used for the To:
> on the tracer. That lets people send the email manually if they want
> to.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/karl3/+bug/770428/+subscribe
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/770428
>
> Title:
> Develop email-in tracer tool
>
> Status in KARL3:
> New
>
> Bug description:
> We don't have a reliable way to know when email-in is wedged. Rather
> than trying to monitor each part of the end-to-end facility, we should
> instead monitor the failure of a tracer email to be received.
>
> The idea is that something, somewhere (hopefully inside the OSF NYC
> LAN) sends a tracer email to KARL every N minutes. This won't be a
> blog entry, as we don't want some dummy blog filling up and the
> content feeds getting blasted.
>
> Instead, we'll develop a new tool, either as part of a community or
> something hanging off the top. This tool will record receipt of the
> email. We'll also make sure that if the tracer wasn't received in the
> last N minutes, monitoring gets triggered.
>
> Some notes:
>
> - Exercise as much of the machinery as possible. We don't really need
> to test the ability to create a blog entry or blog comment. If it
> goes out of the NYC network, through the Internets tubes, into the
> outer Postfix, through its spam filters, to the inner, through
> repoze.postoffice, and is picked up by the daemon, that will handle
> 99% of the problems.
>
> - Make sure admins like Robert/Nat/Paul can go and see something about
> the tracer. Most likely the admin screen.
>
> - Document, in that tool, the email address that is used for the To:
> on the tracer. That lets people send the email manually if they want
> to.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/karl3/+bug/770428/+subscribe

Maybe 4 hours to make the special-purpose tool, done in a way that makes sense.  (We have some more things to discuss on this.)  Probably 1 hour to make the error monitor report it, then an hour to get all parts of it into place.  Add padding as appropriate.

--Paul

On May 10, 2011, at 11:59 AM, Nat Katin-Borland wrote:

> Good point about that much content potentially impacting search.  About
> how much time would we need to implement something like this?  Then we
> can run it by Tom.  I don't want to wait too long to have something in
> place because we've already had 2 periods of down time in the last 6
> days and we're going to start to loose users if email isn't reliably
> delivered.
> 
> -Nat
> 
> --
> Nathaniel Katin-Borland
> Support Specialist
> Knowledge Management Initiative
> KARL Support Team
> 
> Open Society Foundations - New York Office
> 400 West 59th Street
> New York, NY 10019
> Email: nborland@sorosny.org
> Phone: 212-547-6984
> http://www.soros.org/
> http://www.karlproject.org 
> 
> -----Original Message-----
> From: bounces@canonical.com [mailto:bounces@canonical.com] On Behalf Of Chris Rossi
> Sent: Tuesday, May 10, 2011 11:18 AM
> To: Nathaniel Katin-Borland
> Subject: Re: [Bug 770428] Re: Develop email-in tracer tool
> 
> On Tue, May 10, 2011 at 10:59 AM, Robert Marianski <
> 770428@bugs.launchpad.net> wrote:
> 
>> I was assuming these tracer emails would just be blog post entries, 
>> and then the test would be to verify that unique text, like a uuid or 
>> timestamp, was present on the page.
>> 
>> What are the negative consequences of just sending an email to a 
>> separate private community? If we want to eliminate the content feeds 
>> side effect, we can create a separate dummy user just for this tracer 
>> private community.
>> 
>> I guess this makes the test to verify that the mail went through more 
>> complicated, but it might be easier to move the complexity there than 
>> to build something separate in karl.
>> 
>> 
> Well, actually, now that I remember, what we had actually discussed is a specialized blog tool in a specialized community.  So, actually, all of the mailin machinery would be exercised but you wouldn't end up with an enormous number of blog entries stored unnecessarily in your database.  If we're sending a tracer email every 5 minutes, that's 288 blog posts a day.  After just a week, your test community has 2016 blog posts in it, and growing.
> That just doesn't seem sustainable.
> 
> So the idea was to just create an object that looks for all the world
> like a blog for the mailin tool to write a blog entry to, but instead of
> storing each blog entry it would simply not the time it had last
> received a blog entry.  So you still have a bunch of transactions, but
> at least not a ton of data you're trying to store unnecessarily.  Zagy's
> idea also eliminates the unnecessary transactions, but by using this
> dummy blog tool, we still exercise all of the mailin machinery end to
> end.
> 
> Chris
> 
> --
> You received this bug notification because you are a direct subscriber of the bug.
> https://bugs.launchpad.net/bugs/770428
> 
> Title:
>  Develop email-in tracer tool
> 
> Status in KARL3:
>  New
> 
> Bug description:
>  We don't have a reliable way to know when email-in is wedged.  Rather
>  than trying to monitor each part of the end-to-end facility, we should
>  instead monitor the failure of a tracer email to be received.
> 
>  The idea is that something, somewhere (hopefully inside the OSF NYC
>  LAN) sends a tracer email to KARL every N minutes.  This won't be a
>  blog entry, as we don't want some dummy blog filling up and the
>  content feeds getting blasted.
> 
>  Instead, we'll develop a new tool, either as part of a community or
>  something hanging off the top.  This tool will record receipt of the
>  email.  We'll also make sure that if the tracer wasn't received in the
>  last N minutes, monitoring gets triggered.
> 
>  Some notes:
> 
>  - Exercise as much of the machinery as possible.  We don't really need
>  to test the ability to create a blog entry or blog comment.  If it
>  goes out of the NYC network, through the Internets tubes, into the
>  outer Postfix, through its spam filters, to the inner, through
>  repoze.postoffice, and is picked up by the daemon, that will handle
>  99% of the problems.
> 
>  - Make sure admins like Robert/Nat/Paul can go and see something about
>  the tracer. Most likely the admin screen.
> 
>  - Document, in that tool, the email address that is used for the To:
>  on the tracer.  That lets people send the email manually if they want
>  to.
> 
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/karl3/+bug/770428/+subscribe
> 
> -- 
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/770428
> 
> Title:
>  Develop email-in tracer tool
> 
> Status in KARL3:
>  New
> 
> Bug description:
>  We don't have a reliable way to know when email-in is wedged.  Rather
>  than trying to monitor each part of the end-to-end facility, we should
>  instead monitor the failure of a tracer email to be received.
> 
>  The idea is that something, somewhere (hopefully inside the OSF NYC
>  LAN) sends a tracer email to KARL every N minutes.  This won't be a
>  blog entry, as we don't want some dummy blog filling up and the
>  content feeds getting blasted.
> 
>  Instead, we'll develop a new tool, either as part of a community or
>  something hanging off the top.  This tool will record receipt of the
>  email.  We'll also make sure that if the tracer wasn't received in the
>  last N minutes, monitoring gets triggered.
> 
>  Some notes:
> 
>  - Exercise as much of the machinery as possible.  We don't really need
>  to test the ability to create a blog entry or blog comment.  If it
>  goes out of the NYC network, through the Internets tubes, into the
>  outer Postfix, through its spam filters, to the inner, through
>  repoze.postoffice, and is picked up by the daemon, that will handle
>  99% of the problems.
> 
>  - Make sure admins like Robert/Nat/Paul can go and see something about
>  the tracer. Most likely the admin screen.
> 
>  - Document, in that tool, the email address that is used for the To:
>  on the tracer.  That lets people send the email manually if they want
>  to.
> 
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/karl3/+bug/770428/+subscribe