emit a callback on oops, for realtime reporting / graphing

[Selene ToyKeeper]

I would like wsgi-oops to emit a signal of some sort or call a callback whenever it completes an oops report, so I can hook up real-time monitoring tools. Ideally, I would like to graph the frequency of errors with a resolution of at least one pixel per 5 minutes, and potentially send alerts if the average rate spikes.

I'd like to send the data to a system which is not oops-specific, so I can include other events too. For example, logging the timestamp whenever any software gets deployed, to help correlate releases with error rates.

This article shows an example of what I have in mind:

  http://codeascraft.etsy.com/2010/12/08/track-every-release/

[Robert Collins (lifeless)]

That would be ok; personally I'd like to make the oops be transmitted
in real time, then we can emit the signals from the receiver, rather
than each appserver (and so only need one implementation of the signal
emitter) (vs the 3 or more oops implementations out there).

[Selene ToyKeeper (toykeeper)]

That would be even better. I didn't think the realtime oops transmission was going to happen until OOPS2 though.

There is a similar but more general discussion happening on launchpad-dev, spurred by another Etsy article and this week's data science sprint. If we start using a similar system, it sounds like I'm not the only one who wants to tie in realtime oops data. It doesn't have to be in wsgi-oops though, so long as it goes in somewhere.