bounce mail processed multiple times -> oom crash of BounceRunner
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
GNU Mailman |
Fix Released
|
Medium
|
Mark Sapiro |
Bug Description
System: RHEL 7.7
Mailman: mailman-
Description: We have 2 Mailinglist with > 7000 Members which trigger a spike in processed bounces (more than 3000 in an sinle run), followed by an Out of Memory situation in the BounceRunner and an > 20 GB bounce-
file.
We tried to mitigate the problem by increasing the system memory, running the BoucneRunner
every minute and limiting the number of mails delivered at ounce by postfix.
But it happened again:
Dec 30 19:53:29 2019 (13392) <BounceRunner at 140395473885088> processing 4134 queued bounces
...
Dec 30 19:53:59 mx09 kernel: [13392] 41 13392 2755797 1874474 5337 825695 0 python
...
Dec 30 19:53:59 mx09 kernel: Out of memory: Kill process 13392 (python) score 896 or sacrifice child
Dec 30 19:53:59 mx09 kernel: Killed process 13392 (python), UID 41, total-vm:
We analyzed the bounce-event file, extracting data with "stings".
This time we extracted the postfix mail queue ids from the received headers
with our listserver. We found the following:
cat /tmp/bounce-
1 01A7DE9314
1 10F6AE9319
1 18456E930E
1 27D0BAC960
1 3B51CE9316
1 57C2DAC992
1 5D3B2E9310
1 5EF11E9311
1 63054E9312
1 69377E9313
1 ED636E930F
2 29884E9315
2 49ECEAC98D
2 99A16A9DA7
2 (Postfix)
3 59EE1AC995
3 CEB61AC996
192 C12E9D3B48
3929 F2BEEE9318
4122 CC58AAC993
4134 6EFD1A9DA7
As the bounces the last 2 qids are from the original mail send to the list (6EFD1A9DA7),
and one of mails send by mailman to 500 members of that list (CC58AAC993).
(F2BEEE9318 and C12E9D3B48) are both bounces from members that where deliverd only once
to /usr/lib/
So from me it seems that somehow some few bounce where "multiplied" so that ~20 real
bounce produced 4134 virtual bounce.
I see the potential of a deny of service attack, as it could be used
to fill up the disk where the bounce-event files get dumped
But I don't know if this would warrant marking "This bug is a security vulnerability".
Related branches
Changed in mailman: | |
status: | In Progress → Fix Committed |
Changed in mailman: | |
status: | Fix Committed → Fix Released |
We need more information to know what's happening.
You say the MTA log shows only one delivery for, e.g., F2BEEE9318, to /usr/lib/ mailman/ mail/mailman bounces NAME-OF-HUGE-LIST yet strings on the bounce event file finds F2BEEE9318 a total of 3929 times. Can you examine the bounce- events- ppppp.pck file with Mailman's bin/dumpdb and see what the actual bounce messages are and where they all come from?