mailman breaks CC field incorrectly when spread over multiple lines

Bug #659975 reported by Satish P on 2010-10-13
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
mailman (Debian)
New
Unknown
mailman (Ubuntu)
Low
James Page
Hardy
Low
James Page

Bug Description

--------- Information for SRU ---------
IMPACT:
The version of mail in 8.04 does not parse email addresses which are broken over a '\n' correctly; as a result two emails are generated by the AvoidDuplicates handler which is not correct:

Cc sent in email:

Cc: <email address hidden>, "user two" <email address hidden>, user
          three <email address hidden>

Cc generated by AvoidDuplicates handler:

Cc: three <email address hidden>,
 <email address hidden>, <email address hidden>

This is due to a bug in the bundled version (2.5.8) of the Python email package that is distributed with this version of mailman.

FIX:
The _parseaddr.py function has been extracted from the bundled version of the Python email package and patched to resolve this issue. This brings this function in-line with later versions of Python.

This file resides in the debian directory is overlaid on-top of the extracted tar.gz during the package build process.

In actual fact the version of Python distributed on 8.04 already contains this fix; however I believe this approach is the least risk as it is the most light touch.

PATCH:
See linked branch

TEST CASE:
Using a standalone mailman + postfix setup on Hardy use the following procedure (generated from Ubuntu Server Guide):

  Install postfix and configure as final destination for <hostname>.example.com
  Setup three users (user1, user2 and user3)
  Install mailman and setup default (mailman) and test-list (with user1 as list owner) lists.
  Configure Apache to allow administration of mail lists.
  Install and configure dovecot for local mail delivery.
  Add user2 to the test-list mail list using the administration screens.

Using the attached email (test.txt), simulate the offending use case using the following command using the user1 account:

  telnet localhost 25 < test.txt

REGRESSION POTENTIAL:
If other functions within mailman work around this bug they may be impacted by this fix. Due to the bundling of this package the impact should be limited to the mailman application.

--------- Information for SRU ---------

ORIGINAL BUG REPORT:

Binary package hint: mailman

Scenario:

When we send an email to a mailing list in either To: or Cc: header and an address in Cc header with unquoted display name, mailman breaks that into couple of addresses. For example:

To: <email address hidden>
Cc: <email address hidden>, display name <email address hidden>

It becomes:

Cc: <email address hidden>, display, name <email address hidden>

This is wrong because "display" becomes a different address. There seems to be bug in /usr/lib/mailman/pythonlib/email/_parseaddr.py.

Same bug is reported in Debian and Redhat as well for 2.1.9 version of mailman.

https://partner-bugzilla.redhat.com/show_bug.cgi?id=603635
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=516300

System Information:

lsb_release -rd
Description: Ubuntu 8.04.3 LTS
Release: 8.04

dpkg -l | grep -E '(mailman|python)'
ii mailman 1:2.1.9-9ubuntu1 Powerful, web-based mailing list manager
ii python 2.5.2-0ubuntu1 An interactive high-level object-oriented la
ii python-apt 0.7.4ubuntu7.5 Python interface to libapt-pkg
ii python-central 0.6.7ubuntu0.1 register and build utility for Python packag
ii python-gdbm 2.5.2-0ubuntu2 GNU dbm database support for Python
ii python-gnupginterface 0.3.2-9ubuntu1 Python interface to GnuPG (GPG)
ii python-minimal 2.5.2-0ubuntu1 A minimal subset of the Python language (def
ii python-support 0.7.5ubuntu1 automated rebuilding support for python modu
ii python2.5 2.5.2-2ubuntu6 An interactive high-level object-oriented la
ii python2.5-minimal 2.5.2-2ubuntu6 A minimal subset of the Python language (ver

Actual Results:

Single email address in Cc header became two addresses

Expected Results:

Single email address should remain as one address only

Related branches

James Page (james-page) wrote :

This version of mailman bundles the python email package for compatibility; the patch will need to be applied within the email-2.5.8.tar.gz located in this package.

Changed in mailman (Ubuntu):
importance: Undecided → Low
status: New → Confirmed
James Page (james-page) wrote :

Discussed this bug with the guys on #ubuntu-server and I think that its worth raising a SRU for this bug.

I'll work on a patch to feed into this process; however it will need extensive testing - @Satish is this something that you would be able to help out with?

Changed in mailman (Ubuntu):
assignee: nobody → James Page (james-page)
status: Confirmed → In Progress
James Page (james-page) wrote :

I'm having trouble re-producing this issue on Hardy; please could you detail exactly how this can be tested and what combination of mailman and mail transfer agent you are using (postfix/exim etc...)

Thanks

Changed in mailman (Ubuntu):
status: In Progress → Incomplete
assignee: James Page (james-page) → nobody
Changed in mailman (Debian):
status: Unknown → New
Satish P (spulikon) wrote :

We are using postfix (version 2.5.1-2ubuntu1.2) and mailman version 2.1.9-9ubuntu1 in our environment. The scenario in which we noticed this problem:

To: <email address hidden>
Cc: <email address hidden>, <email address hidden>, <email address hidden>, <email address hidden>, display
      name <email address hidden>

There was an email sent by an user as mentioned above. In this scenario, display and name <email address hidden> became two email addresses (as noticed by the user who received the email from mailing list).

Yes, I am willing to test the patches.

James Page (james-page) wrote :

OK; the key bit I missed was that the incorrectly parsed email address was broken over two lines.

Using a standalone mailman + postfix setup on Hardy I have been able to reproduce using the following procedure:

  Install postfix and configure as final destination for <hostname>.example.com
  Setup three users (user1, user2 and user3)
  Install mailman and setup default (mailman) and test-list (with user1 as list owner) list.
  Configure Apache to allow administration of mail lists
  Install and configure dovecot for local mail delivery.
  Add user2 to the test-list mail list using the administration screens

Using the attached email (test.txt), simulate the offending use case using the following command using the user1 account:

  telnet localhost 25 < test.txt

The posting to the test-list mailing list incorrectly has two emails addresses:

Date: Mon, 18 Oct 2010 10:44:12 +0100 (BST)
From: <email address hidden>
Cc: three <email address hidden>,
 <email address hidden>, <email address hidden>
Subject: [Test-list] Test

Test Email
_______________________________________________
Test-list mailing list
<email address hidden>
http://hardy-mailman.example.com/cgi-bin/mailman/listinfo/test-list

James Page (james-page) on 2010-10-18
Changed in mailman (Ubuntu):
status: Incomplete → Confirmed
status: Confirmed → In Progress
James Page (james-page) on 2010-10-18
description: updated
James Page (james-page) on 2010-10-18
description: updated
summary: - mailman breaks CC field incorrectly
+ mailman breaks CC field incorrectly when spread over multiple lines
description: updated
James Page (james-page) wrote :

Note that this bug only impacts the mailman package up to and including 2.1.9-9ubuntu1 (as distributed on 8.04) as the bundled version of Python email is removed in later versions

Changed in mailman (Ubuntu):
assignee: nobody → James Page (james-page)
Martin Pitt (pitti) wrote :

Why does this ship an entirely new file instead of a patch? Is there a patch to review here? Thanks!

Martin Pitt (pitti) wrote :

Closing main Ubuntu task as per comment 6.

Changed in mailman (Ubuntu):
status: In Progress → Fix Released
Changed in mailman (Ubuntu Hardy):
status: New → In Progress
assignee: nobody → James Page (james-page)
James Page (james-page) wrote :

I chose this approach as the fix required needs to be applied within misc/email-3.5.8.tar.gz which is expanded during the package build process.

I can rework this into a patch for the _parseaddr.py contained with misc/email-3.5.8.tar.gz and patch instead of overwriting if you would like me to.

Martin Pitt (pitti) wrote :

James,

ah, that makes sense. You can keep the package as it is, I am just interested in seeing a diff of what will actually change. Thanks!

James Page (james-page) wrote :

As requested - patch detailing the changes made to _parseaddr.py.

Satish P (spulikon) wrote :

James,

Is the mailman package updated in ubuntu? If yes, where can I d/l the updated one?

Thanks for fixing this.

Martin Pitt (pitti) wrote :

James, that looks fine, thanks! Please go ahead and get this uploaded.

James Page (james-page) on 2010-10-22
Changed in mailman (Ubuntu Hardy):
importance: Undecided → Low
James Page (james-page) on 2010-11-04
Changed in mailman (Ubuntu Hardy):
assignee: James Page (james-page) → nobody
James Page (james-page) on 2010-11-04
Changed in mailman (Ubuntu Hardy):
assignee: nobody → James Page (james-page)
Sebastien Bacher (seb128) wrote :

Chuck you seem to have worked on this package before, could you sponsor the sru there if it makes sense to you?

Mathias Gug (mathiaz) wrote :

Looks good to me. Uploaded to hardy-proposed.

Accepted mailman into hardy-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in mailman (Ubuntu Hardy):
status: In Progress → Fix Committed
tags: added: verification-needed
James Page (james-page) wrote :

Looks good on my virtual test instance that I did the original confirmation and fix on.

Satish - are you able to support with more in-depth testing?

Thanks

Satish P (spulikon) wrote :

I have downloaded the package and right now we are testing in a development server. Thanks for the fix. I will post about results of our testing.

Imre Gergely (cemc) wrote :

I've tested the package from -proposed according to the test case and it seems to work. Below some details.

- set up test-list with user1 and user2 as test users, and user3 NOT a member
- sent a mail to the list, like this:

HELO localhost
MAIL FROM: <email address hidden>
RCPT TO: <email address hidden>
DATA
Subject: Test
Cc: <email address hidden>, "user two" <email address hidden>, user
          three <email address hidden>

Test Email
.
QUIT

- got the following header after it got delivered to user1 (which is a member):

From: <email address hidden>
Cc: <email address hidden>, <email address hidden>,
        three <email address hidden>
Subject: [Test-list] Test

- notice the user@ ... email address which is wrong

After activating -proposed and installed 2.1.9-9ubuntu1.1, sent the same message and got the correct header:

From: <email address hidden>
Cc: user three <email address hidden>,
        <email address hidden>
Subject: [Test-list] Test

I can confirm that the package from -proposed fixes this bug.

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mailman - 1:2.1.9-9ubuntu1.1

---------------
mailman (1:2.1.9-9ubuntu1.1) hardy-proposed; urgency=low

  * Patched bundled python-2.5.8 email package to correctly parse
    email addresses (LP: #659975)
 -- James Page <email address hidden> Thu, 14 Oct 2010 15:43:11 +0100

Changed in mailman (Ubuntu Hardy):
status: Fix Committed → Fix Released
tags: added: testcase
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.