Mail generating multiple Message-ID and sending lots of messages

[Paul Everitt]

Work to be done
===================

- Write some tests
- Put in a config knob with a counter as a throttle, set to 50 initially
- Logging that includes From, original message id, to, subject, and outgoing message id count
- Log an error if the outgoing message id count is higher than 1
- (Can't do this without big fork of method) Catch unhandled SMTP exceptions
- (Central in only_once) Stop logging so much noise on regular mailout usage (across 3 app servers)

Detail
=========

We are getting SMTP server error responses saying that we are being throttled, sending too many emails too quickly. FlyingCircus (Stefan) replied with some analysis:

"""
below a list of conspicuous facts from the mail logs:
the only affected instance is osfkarl10, all other app instances worked as expected
this also affected the outages at 13:07, 16:08, 19:09 and 22:09
the outages iterated every 3 hours
osfkarl10 suddenly started to send tons of mails (one recipient per mail)
for every additional mail osfkarl10 started to deliver it added one more message-id header
first mail:
9C766100107: message-id=: 20150902100612.5138.505965863@osfkarl10
second mai:
E4240100718: message-id=: 20150902100612.5138.505965863@osfkarl10
E4240100718: message-id=: 20150902100612.5138.481089980@osfkarl10
third mail:
3D2491007A6: message-id=: 20150902100612.5138.505965863@osfkarl10
3D2491007A6: message-id=: 20150902100612.5138.481089980@osfkarl10
3D2491007A6: message-id=: 20150902100612.5138.1950918356@osfkarl10
...
the last mail delivered contained 100 message-id headers
at the 101th mail postfix reached its parallel connection limit and refused to accept another mail
at all, osfkarl10 tried to deliver 671 mails
all this happened between 10:06:21 and 10:06:55
refusing to process further messages started 10:06:51
after refusing to process further messages new messages reached at a constant rate of 100mails / second
"""

[Paul Everitt (paul-agendaless)]

Nat: this one is a bug in the widgets that we use:

  http://bugs.jqueryui.com/ticket/8844

The fix they used, though, makes things ugly in IE7. And so it begins...do we work extra hard to support IE7?