Messages can take days to appear in the MhonArc archive
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
lp-mailman |
Triaged
|
High
|
Unassigned |
Bug Description
User have noted that a message they receive from an Lp mailing list may take hours or days appear in the archive. It was observed that there can be a backlog of messages in mailman/
This issue appears to be caused by giant archives, which need to regenerate all the index pages and many messages when a new message is added. Lp indexes are in reverse order, newest to oldest, so when a message is added, a message is pushed to the previous index. Process names on the server imply a lot of time is spent adding messages to https:/
Lp could consider changing the index order for large lists so less work in needed.
Another route is modify Lp's mailman configuration to process a larger slice or archive messages. Lp processes one message at at a time in each queue, but it clearly takes longer to archive one message than it does to send a 1000 messages. The archive qrunner could process 2 or 3 messages to keep up, but this will in fact slow down the other queue slightly.
tags: | added: mailing-lists |
tags: |
added: ml-archive-sucks removed: ml-archives-sucks |
tags: | removed: trivial |
tags: | added: feature |
affects: | launchpad → lp-mailman |
We can improve the queuing issue by updating the QRUNNERS tuple in lp/services/ mailman/ monkeypatches. py
lib/
to a multiple of 2 (only documented in mailman itself). eg:
('ArchRunner', 4)
will archive 4 messages per sent message.
test_mm_ cfg.TestMMCfgDe faultsTestCase. test_qrunners needs revision. The current test only verifies that the queue is enabled. We may want to verify that we gave it a larger slice.