stops processing new mail for a list after receiving attachment with ' in filename

[Steve Alexander]

Today, the launchpad development list, hosted on the Canonical server lists.canonical.com, stopped sending out email. Mail was accepted, but not sent on.

New messages were "shunted" by Mailman.

Here's a relevant part of the traceback.

  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 361, in save_attachment
    fnext = os.path.splitext(msg.get_filename(''))[1]
  File "/usr/lib/python2.4/email/Message.py", line 707, in get_filename
    filename = self.get_param('filename', missing, 'content-disposition')
  File "/usr/lib/python2.4/email/Message.py", line 590, in get_param
    for k, v in self._get_params_preserve(failobj, header):
  File "/usr/lib/python2.4/email/Message.py", line 537, in _get_params_preserve
    params = Utils.decode_params(params)
  File "/usr/lib/python2.4/email/Utils.py", line 275, in decode_params
    charset, language, value = decode_rfc2231(EMPTYSTRING.join(value))
  File "/usr/lib/python2.4/email/Utils.py", line 222, in decode_rfc2231
    charset, language, s = parts
ValueError: need more than 2 values to unpack

The bug is actually in the email package of the python standard library. It is failing to properly handle the contents of the Content-Disposition: header when it contains a single quote character in the filename. This is called when the code msg.get_filename() or msg.get_filename('') in Mailman's Scrubber.py is run.

If this problem is hacked around, you get another traceback of the same issue in a different place.

  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 240, in process
    url = save_attachment(mlist, part, dir)
  File "/var/lib/mailman/Mailman/Handlers/Scrubber.py", line 388, in save_attachment
    filename = msg.get_filename()
  File "/usr/lib/python2.4/email/Message.py", line 707, in get_filename
    filename = self.get_param('filename', missing, 'content-disposition')
  File "/usr/lib/python2.4/email/Message.py", line 590, in get_param
    for k, v in self._get_params_preserve(failobj, header):
  File "/usr/lib/python2.4/email/Message.py", line 537, in _get_params_preserve
    params = Utils.decode_params(params)
  File "/usr/lib/python2.4/email/Utils.py", line 275, in decode_params
    charset, language, value = decode_rfc2231(EMPTYSTRING.join(value))
  File "/usr/lib/python2.4/email/Utils.py", line 222, in decode_rfc2231
    charset, language, s = parts
ValueError: need more than 2 values to unpack

Hacking around this one fixed the issue on the Canonical servers. However, the call to get_filename() is also present in other code paths, apparently when the atachment is not multi-part MIME.

I'll attach a patch that works around all three cases.

[Steve Alexander (stevea)]

This patch catches ValueError exceptions from the email package of the standard library and allows Mailman to continue with an appropriate fall-back value.

[Martin Pitt (pitti)]

It took a while to sort everything out with upstream, but now the issue is public (CVE-2006-2941). There is a new python email package which is supposed to fix this bug.

[Martin Pitt (pitti)]

Fixed edgy:

 mailman (1:2.1.8-2ubuntu2) edgy; urgency=low
 .
   * SECURITY UPDATE: XSS.
   * Add debian/patches/security-CVE-2006-3636-XSS.dpatch:
     - Fix various cross-site scripting vulnerabilities.
     - Patch backported from svn head, thanks to Barry Warsaw for preparing it.
     - CVE-2006-3636
   * Add debian/patches/security-CVE-2006-2941.dpatch:
     - Scrubber.py: Do not bail out if emails' get_filename() throws a
       ValueError. This has been properly fixed in the next upstream email
       package (in Python core), but the fix is very intrusive. Thanks to Steve
       Alexander for discovering this and for the proposed patch.
     - CVE-2006-2941
     - Closes: LP#49620
   * Add debian/patches/security-error_log.dpatch:
     - Check characters in URL to prevent injecting bogus messages into
       error_log.
     - Patch taken from upstream SVN:
       http://svn.sourceforge.net/viewvc/mailman?view=rev&revision=7918

stables fixed uploaded, pending build and publishing.

[Martin Pitt (pitti)]

Fixed stables some days ago in USN-345-1.