mailman crash for subscription in webinterface

[Sebastian Wagner]

A Traceback from mailman's logs for a subscription. The bug can be triggered with the following post-data:
language=''

Jul 11 20:29:34 2016 admin(29209): @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
admin(29209): [----- Mailman Version: 2.1.16 -----]
admin(29209): [----- Traceback ------]
admin(29209): Traceback (most recent call last):
admin(29209): File "/var/lib/mailman/scripts/driver", line 117, in run_main
admin(29209): main()
admin(29209): File "/var/lib/mailman/Mailman/Cgi/subscribe.py", line 73, in main
admin(29209): language = cgidata.getvalue('language')
admin(29209): File "/usr/lib/python2.7/cgi.py", line 548, in getvalue
admin(29209): if key in self:
admin(29209): File "/usr/lib/python2.7/cgi.py", line 594, in __contains__
admin(29209): raise TypeError, "not indexable"
admin(29209): TypeError: not indexable
admin(29209): [----- Python Information -----]
admin(29209): sys.version = 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2]
admin(29209): sys.executable = /usr/bin/python
admin(29209): sys.prefix = /usr
admin(29209): sys.exec_prefix = /usr
admin(29209): sys.path = ['/var/lib/mailman/pythonlib', '/var/lib/mailman', '/usr/lib/mailman/scripts', '/var/lib/mailman', '/usr/lib/python2.7/', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/site-packages']
admin(29209): sys.platform = linux2
admin(29209): [----- Environment Variables -----]
admin(29209): SSL_VERSION_INTERFACE: mod_ssl/2.4.7
admin(29209): HTTP_REFERER: https://mail.example.com/cgi-bin/mailman/subscribe/news
admin(29209): SSL_CIPHER_EXPORT: false
admin(29209): CONTEXT_DOCUMENT_ROOT: /usr/lib/cgi-bin/
admin(29209): SERVER_SOFTWARE: Apache
admin(29209): CONTEXT_PREFIX: /cgi-bin/
admin(29209): SSL_SERVER_A_KEY: rsaEncryption
admin(29209): QUERY_STRING:
admin(29209): SERVER_SIGNATURE:
admin(29209): REQUEST_METHOD: POST
admin(29209): PATH_INFO: /news
admin(29209): SERVER_PROTOCOL: HTTP/1.1
admin(29209): SSL_SERVER_S_DN: CN=mail.example.com
admin(29209): SSL_CIPHER: ECDHE-RSA-AES128-GCM-SHA256
admin(29209): SSL_SERVER_V_START: Apr 17 16:42:00 2016 GMT
admin(29209): SSL_TLS_SNI: mail.example.com
admin(29209): CONTENT_LENGTH: 106
admin(29209): SSL_CLIENT_VERIFY: NONE
admin(29209): HTTP_USER_AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0
admin(29209): HTTP_CONNECTION: keep-alive
admin(29209): HTTP_COOKIE: PHPSESSID=...
admin(29209): SERVER_NAME: mail.example.com
admin(29209): REMOTE_ADDR: 192.0.2.1
admin(29209): SSL_CIPHER_ALGKEYSIZE: 128
admin(29209): SSL_SECURE_RENEG: true
admin(29209): PATH_TRANSLATED: /srv/web/mail/news
admin(29209): SSL_SERVER_I_DN_C: US
admin(29209): SSL_COMPRESS_METHOD: NULL
admin(29209): SSL_SERVER_M_VERSION: 3
admin(29209): SSL_SERVER_I_DN_O: Let's Encrypt
admin(29209): SERVER_ADDR: 192.0.2.2
admin(29209): DOCUMENT_ROOT: /srv/web/mail
admin(29209): SERVER_PORT: 443
admin(29209): SSL_VERSION_LIBRARY: OpenSSL/1.0.1f
admin(29209): PYTHONPATH: /var/lib/mailman
admin(29209): SCRIPT_FILENAME: /usr/lib/cgi-bin/mailman/subscribe
admin(29209): SERVER_ADMIN: <email address hidden>
admin(29209): SSL_SESSION_RESUMED: Initial
admin(29209): SSL_SERVER_M_SERIAL: ...
admin(29209): SSL_SERVER_A_SIG: sha256WithRSAEncryption
admin(29209): HTTP_DNT: 1
admin(29209): HTTP_HOST: mail.example.com
admin(29209): SCRIPT_NAME: /cgi-bin/mailman/subscribe
admin(29209): HTTPS: on
admin(29209): HTTP_CACHE_CONTROL: max-age=0
admin(29209): REQUEST_URI: /cgi-bin/mailman/subscribe/news
admin(29209): HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
admin(29209): SSL_SERVER_S_DN_CN: mail.example.com
admin(29209): GATEWAY_INTERFACE: CGI/1.1
admin(29209): SSL_SERVER_I_DN_CN: Let's Encrypt Authority X3
admin(29209): REMOTE_PORT: 40456
admin(29209): HTTP_ACCEPT_LANGUAGE: en-US,en;q=0.5
admin(29209): REQUEST_SCHEME: https
admin(29209): SSL_SERVER_V_END: Jul 16 16:42:00 2016 GMT
admin(29209): CONTENT_TYPE: text/plain;charset=UTF-8
admin(29209): SSL_PROTOCOL: TLSv1.2
admin(29209): SSL_CIPHER_USEKEYSIZE: 128
admin(29209): HTTP_ACCEPT_ENCODING: gzip, deflate, br
admin(29209): SSL_SERVER_I_DN: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US

[Sebastian Wagner (sebix)]

Launchpad didn't gave be an opportunity to fill in version information, thus here as a comment:
mailman 1:2.1.16-2ubuntu0.1
Ubuntu 14.04.4 LTS

[Mark Sapiro (msapiro)]

I see how this can occur if you have your own page that POSTs to the subscribe CGI with no post data or post data consisting of all blank values, but I don't see how it occurs with a post from the listinfo subscribe form.

Can you explain exactly what the scenario is that triggers this error?

[Sebastian Wagner (sebix)]

Does it matter where the request comes from? Anyone can send post-requests to the mailman page (subscribers, bots, hackers etc) and mailman should not fail in any case IMHO.

> Can you explain exactly what the scenario is that triggers this error?
This is the minimal example I could find:
curl https://mail.example.com/cgi-bin/mailman/subscribe/news -d "language='" -H "Content-Type: text/plain;charset=UTF-8"
You can of course add as many other data as you want.

[Mark Sapiro (msapiro)]

Actually, the problem is both simpler and more wide spread than you report. The underlying issue is the Content-Type: text/plain header sent with the POST request. It doesn't matter what if anything the data is. 'language' is a red herring. It only appears in the tracebacks from the subscribe CGI because that is what's being requested in the subscribe CGI's first call to the getvalue method of the cgi.FieldStorage instance.

The way the Python cgi module works, the FieldStorage instance has different properties depending on the Content-Type: header in the POST. If the content type is application/x-www-form-urlencoded as would be the case in a normal POST from a browser the FieldStorage instance is a dictionary-like mapping of key, value pairs that can be retrieved via the getvalue method. If the content type is text/plain, the FieldStorage instance just has a string value and the getvalue method throws TypeError.

This actually affects every one of Mailman's CGIs, not just subscribe, and without knowing how the error was triggered, I probably wouldn't have determined the cause.

In the spirit of not throwing uncaught exceptions, even when people, web crawlers, etc. unwittingly or maliciously craft defective requests, I have committed http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1663 to return an error document with a 400 status for such requests.

[Mark Sapiro (msapiro)]

The fix for https://bugs.launchpad.net/bugs/1614841 caused a regression of this fix in options.py. This regression is fixed in http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1711.