mailman breaks after tklbam 1.1 and turnkey-pylib 0.3 upgrade

[Jochem Berends]

See also the forumpost: http://www.turnkeylinux.org/forum/support/20120304/mailman-breaks-right-after-turnkey-pylib-upgrade-any-correlation

Mailmain breaks since Mar 1st ~ 14:00. It is installed on a TKL Redmine installation (11.2 I believe). The error messages that I receive since that time are:

/var/log/mail.info:Mar 1 14:28:35 redmine postfix/pipe[27195]: B6EE3200C4: to=<email address hidden>,
relay=mailman, delay=3.9, delays=0.17/0.04/0/3.7, dsn=5.3.0, status=bounced (Command died with
status 1: "/usr/lib/mailman/bin/postfix-to-mailman.py". Command output:
Traceback (most recent call last):
   File "/usr/lib/mailman/bin/postfix-to-mailman.py", line 91, in <module>
     from Mailman import mm_cfg ImportError: No module named Mailman )

When I look in the cron-apt log I see the following updates over the last week:

libpq-dev libpq5 on Wed Feb 29 12:23:02 UTC 2012
tklbam turnkey-pylib on Thu Mar 1 12:23:01 UTC 2012
tklbam on Fri Mar 2 12:54:57 UTC 2012

Before this time, the Mailman process, spawned by Postfix ran beautifully with a number of lists and with 10ths of msgs a day. After this update of tklbam and turnkey-pylib, (these were the only changes to the system!) The mailman script cannot seem to import its own library. Is it possible that something has been reset in the python paths by the turnkey-pylib?

Also the etcconfig log does not show clear signs of changes surrounding that date and time? Any help appreciated to resolve the matter.

--- update

I removed the turnkey-pylib installed package (version 0.3+31+g3e22221). Naturally, as a result this will break some packages (confconsole tklbam turnkey-sysinfo). These are essential for the backup to function, but not essential for the instance to run.

When sending emails to the list I get good results.

Mar 4 15:54:22 tech postfix/pipe[8716]: 3D0A820119: to=<email address hidden>,
  relay=mailman, delay=1.7, delays=1.2/0.04/0/0.46, dsn=2.0.0,
  status=sent (delivered via mailman service)
The conclusion for me is that turnkey-pylib (or tklbam or related software) breaks something in the python installation due to which mailman does not function anymore.

Basic test is the following command from the commandline:

Correct working example (without turnkey-pylib 0.3+31+g3e22221 installed):

# /usr/lib/mailman/bin/postfix-to-mailman.py
Illegal invocation: '/usr/lib/mailman/bin/postfix-to-mailman.py'

Incorrect example (with turnkey-pylib 0.3+31+g3e22221 installed):

# /usr/lib/mailman/bin/postfix-to-mailman.py
Traceback (most recent call last):
  File "/usr/lib/mailman/bin/postfix-to-mailman.py", line 91, in <module>
    from Mailman import mm_cfg
ImportError: No module named Mailman

When using python debugger on the postfix-to-mailman.py the pythonpath /etc/mailman is not added by the paths.py invocation.

sys.path pythonpath without turnkey-pylib installed
['/var/lib/mailman/pythonlib', '/var/lib/mailman', '/etc/mailman',
'/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk',
'/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload',
'/usr/lib/python2.6/dist-packages', '/usr/lib/pymodules/python2.6',
'/usr/local/lib/python2.6/dist-packages', '/usr/lib/mailman/bin',
'/usr/lib/python2.6/site-packages']

incorrect pythonpath (by paths.py with turnkey-pylib installed)
['/var/lib/mailman/pythonlib', '/var/lib/mailman', '',
'/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '
/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload',
'/usr/lib/python2.6/dist-packages', '/usr/lib/pymodules/python2.6',
'/usr/local/lib/python2.6/dist-packages', '/usr/lib/mailman/bin',
'/usr/lib/python2.6/site-packages']

I will check if I can revert back to an old release of turnkey-pylib (version 0.2+10+gcb57296) and if that seems to work. Right now I have no tklbam because of the removal of turnkey-pylib.

It seems that tklbam 1.1 depends on turnkey-pylib >=0.3 which is difficult for me to test in the production environment as this seems to break postfix_to_mail.py

[Jochem Berends (jberends)]

All is working fine when I downgraded to tklbam 1.0 (requires turnkey-pylib 0.2) and turnkey-pylib 0.2.

http://archive.turnkeylinux.org/ubuntu/pool/lucid/main/t/turnkey-pylib/turnkey-pylib_0.2%2b10%2bgcb57296_i386.deb
and

http://archive.turnkeylinux.org/ubuntu/pool/lucid/main/t/tklbam/tklbam_1.0_all.deb
I set the versions for these two packages on 'hold' until this issue is resolved.

[Jeremy Davis (jedmeister)]

Looks like this regression was introduced whislt fixing this bug: https://bugs.launchpad.net/turnkeylinux/+bug/910515

[Liraz (liraz-siri)]

(copying verbatim from my forum comment)
You're in luck. By happy coincidence we're also mailman users so I experienced this issue myself when I was tested the latest version of turnkey-pylib.

Long story short, edit /etc/mailman/postfix-to-mailman.py and change the line after import sys.os like this:

import sys, os
sys.path.insert(0, "/usr/lib/mailman/bin")
import paths

What's going on? Mailman broke due to a freakish interaction with turnkey-pylib.

turnkey-pylib has a paths.py module. So does mailman. Normally this wouldn't be a problem as local modules are preferred over system modules. But the postfix-to-mailman.py script is installed in /etc/mailman so /usr/lib/mailman/bin/paths.py isn't even in its path.

The postfix-to-mailman.py attempts to correct this by manipulating sys.path but it does so incorrectly by appending /usr/lib/mailman/bin to sys.path instead of inserting it at the beginning.

That's why turnkey-pylib's paths.py overrode mailman's paths.py.