usenet threading improvements

Bug #557955 reported by mdw21
This bug affects 1 person
Affects Status Importance Assigned to Milestone
GNU Mailman
Mark Sapiro

Bug Description

Basically message-id munging for all messages.

Might be
better as an option? This helps with threading in usenet gating, as
what happens is this:

someone sends a mail with message-
id <email address hidden>
gating replaces the message-id with
<email address hidden>
list member replies via mail,
"in-reply-to" header is set to the <email address hidden>
now when the
message is gated it can't thread because in-reply-to: (which is
translated into references:) is wrong.

Solution is to munge
all message-ids. Doesn't really do any harm AFAICT.

Related branches

Revision history for this message
mdw21 (mdw21) wrote :

The file CookHeaders.patch was added: munging code again...

Revision history for this message
elecnix (elecnix-users) wrote :

Logged In: YES

The submitted patch does not do what it should to. The calls
to time.time() will not return the same thing as in I don't know much about the internals of
Mailman, and not much either about Python, so I don't really
know where to store the "correct" message ID so that it is
generated once, and used in both places.

Revision history for this message
elecnix (elecnix-users) wrote :

Logged In: YES

I solved the problem by adding the munging code to
DeliverToList, in, right before the message is
sent in the delivery pipeline, and removed it from and This way, the Message-ID
header is calculated only once, and used for all the steps
of the pipeline.

Nicolas Marchildon

Revision history for this message
mundaun (mundaun) wrote :

Logged In: YES


we had exactly the same problem. However we run mailman
2.1.2 where things seem to have changed, so I am now writing
down the required steps for more recent releases...

I had to copy the munging code from
Mailman/Queue/ to

block 1:
# Matches our Mailman crafted Message-IDs. See
mcre = re.compile(r"""
    <mailman. # match the prefix
    \d+. # serial number
    \d+. # time in seconds since epoch
    \d+. # pid
    (?P<listname>[^@]+) # list's internal_name()
    @ # <email address hidden>
    (?P<hostname>[^>]+) # list's host_name
    > # trailer
    """, re.VERBOSE)

block 2:
    hackmsgid = 1
    if msgid:
        mo =
        if mo:
            lname, hname ='listname', 'hostname')
            if lname == mlist.internal_name() and hname ==
                hackmsgid = 0
    if hackmsgid:
        del msg['message-id']
        msg['Message-ID'] = Utils.unique_message_id(mlist)


After that, my biggest problem was that I applied the changes
but mailman didn't follow them. I finally recognised that I had to
restart the qrunner daemon (which wasn't there in older

Hope this helps
- michael

Revision history for this message
Mark Sapiro (msapiro) wrote :

I don't intend to replace the Message-ID: in as suggested in this patch, but I will put the original Message-ID in References: and perhaps make other modifications in for the posts actually gated to Usenet.

Changed in mailman:
assignee: nobody → Mark Sapiro (msapiro)
milestone: none → 2.1.22
status: New → In Progress
Revision history for this message
Mark Sapiro (msapiro) wrote :

The fix adds the original Message-ID: to References: and also adds X-Mailman-Original-Message-ID: and X-Mailman-Original-References: headers for 'forensics'. Hopefully this will help at least partially mitigate threading issues that result from Message-ID: munging.

Also note that Munging the Message-ID in as suggested here is not a full solution as there still may be direct To: or Cc: recipients from 'reply all' and these are people likely to generate additional replies so the issue of multiple Message-ID:s is not solved by that approach.

Changed in mailman:
status: In Progress → Fix Committed
Mark Sapiro (msapiro)
Changed in mailman:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers