Unicode error prevents delivery of post

Bug #1035057 reported by Stephen Turnbull
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
GNU Mailman
New
Undecided
Unassigned

Bug Description

So here's the build.

######################## Mailman3 build log ########################

# N.B. "Mailman3" is a new, active python 2.7 virtualenv.
# God, do I hate bzr for not being git.

steve@turnbull:~/src/Mailman3/mailman-trunk$ bzr revno
7157
steve@turnbull:~/src/Mailman3/mailman-trunk$ bzr log -r 7157
------------------------------------------------------------
revno: 7157 [merge]
committer: Barry Warsaw <email address hidden>
branch nick: 3.0
timestamp: Thu 2012-07-26 00:22:19 -0400
message:
   * The policy for archiving has now been collapsed into a single enum, called
     ArchivePolicy. This describes the three states of never archive, archive
     privately, and archive_publicly. (LP: #967238)

  Database
  --------
   * Schema migrations (LP: #971013)
     - include_list_post_header -> allow_list_posts
     - news_prefix_subject_too -> nntp_prefix_subject_too
     - news_moderation -> newsgroup_moderation
     - archive and archive_private have been collapsed into archive_policy.
     - nntp_host has been removed.
   * The PostgreSQL port of the schema accidentally added a moderation_callback
     column to the mailinglist table. Since this is unused in Mailman, it was
     simply commented out of the base schema for PostgreSQL.
------------------------------------------------------------
steve@turnbull:~/src/Mailman3/mailman-trunk$ python
Python 2.7.3rc2 (default, Apr 22 2012, 22:30:17)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
steve@turnbull:~/src/Mailman3/mailman-trunk$ python bootstrap.py
Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
Creating directory '/home/steve/src/Mailman3/mailman-trunk/bin'.
Creating directory '/home/steve/src/Mailman3/mailman-trunk/parts'.
Creating directory '/home/steve/src/Mailman3/mailman-trunk/eggs'.
Creating directory '/home/steve/src/Mailman3/mailman-trunk/develop-eggs'.
Getting distribution for 'setuptools'.
Got setuptools 0.6c12dev-r88846.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/buildout'.
steve@turnbull:~/src/Mailman3/mailman-trunk$ bin/buildout
Develop: '/home/steve/src/Mailman3/mailman-trunk/.'
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.8.tar.gz
Extracting in /tmp/tmpOzSp2M
Now working in /tmp/tmpOzSp2M/distribute-0.6.8
Building a Distribute egg in /home/steve/src/Mailman3/mailman-trunk
/home/steve/src/Mailman3/mailman-trunk/distribute-0.6.8-py2.7.egg
warning: no files found matching '*.rc'
warning: no files found matching '*' under directory '.buildout'
warning: no files found matching '*' under directory 'contrib'
warning: no previously-included files matching '*.egg-info' found anywhere in distribution
no previously-included directories found matching 'src/attic'
no previously-included directories found matching 'src/web'
no previously-included directories found matching 'parts'
Getting distribution for 'z3c.recipe.scripts'.
Got z3c.recipe.scripts 1.0.1.
Getting distribution for 'zc.recipe.egg>=1.3.0'.
Got zc.recipe.egg 1.3.2.
Getting distribution for 'z3c.recipe.tag'.
Got z3c.recipe.tag 0.4.1.
Getting distribution for 'zc.recipe.testrunner'.
zip_safe flag not set; analyzing archive contents...
Got zc.recipe.testrunner 1.4.0.
Getting distribution for 'zope.testrunner'.
package init file 'src/zope/testrunner/testrunner-ex/__init__.py' not found (or not a regular file)
Creating missing __init__.py for zope.testrunner.testrunner-ex
Got zope.testrunner 4.0.4.
Getting distribution for 'zope.interface'.
Got zope.interface 4.0.1.
Getting distribution for 'zope.exceptions'.
Got zope.exceptions 4.0.0.1.
Installing interpreter.
Getting distribution for 'zope.testing<4'.
warning: no files found matching '*.test' under directory 'src'
warning: no files found matching 'sampletests' under directory 'src'
Got zope.testing 3.10.3.
Getting distribution for 'zope.event'.
Got zope.event 4.0.0.
Getting distribution for 'zope.configuration'.
Got zope.configuration 4.0.0.
Getting distribution for 'zope.component'.
Got zope.component 4.0.0.
Getting distribution for 'storm'.
Got storm 0.19.
Getting distribution for 'restish'.
Got restish 0.12.1.
Getting distribution for 'passlib'.
warning: no files found matching 'passlib/*.cfg'
no previously-included directories found matching '*.komodoproject'
Got passlib 1.6.1.
Getting distribution for 'mock'.
warning: no files found matching '*.png' under directory 'docs'
warning: no files found matching '*.css' under directory 'docs'
warning: no files found matching '*.html' under directory 'docs'
warning: no files found matching '*.js' under directory 'docs'
zip_safe flag not set; analyzing archive contents...
Got mock 1.0b1.
Getting distribution for 'lazr.smtptest'.
Got lazr.smtptest 1.3.
Getting distribution for 'lazr.config'.
Got lazr.config 1.1.3.
Getting distribution for 'flufl.lock'.
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz
Extracting in /tmp/easy_install-_r_jTR/flufl.lock-2.2.1/temp/tmpUjGygF
Now working in /tmp/easy_install-_r_jTR/flufl.lock-2.2.1/temp/tmpUjGygF/distribute-0.6.10
Building a Distribute egg in /tmp/easy_install-_r_jTR/flufl.lock-2.2.1
/tmp/easy_install-_r_jTR/flufl.lock-2.2.1/distribute-0.6.10-py2.7.egg
zip_safe flag not set; analyzing archive contents...
flufl.__init__: module references __path__
Got flufl.lock 2.2.1.
Getting distribution for 'flufl.i18n'.
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz
Extracting in /tmp/easy_install-HqJWPx/flufl.i18n-1.1.1/temp/tmp33Zt_H
Now working in /tmp/easy_install-HqJWPx/flufl.i18n-1.1.1/temp/tmp33Zt_H/distribute-0.6.10
Building a Distribute egg in /tmp/easy_install-HqJWPx/flufl.i18n-1.1.1
/tmp/easy_install-HqJWPx/flufl.i18n-1.1.1/distribute-0.6.10-py2.7.egg
Got flufl.i18n 1.1.1.
Getting distribution for 'flufl.enum'.
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.19.tar.gz
Extracting in /tmp/easy_install-10gV93/flufl.enum-3.3.2/temp/tmpz01MIV
Now working in /tmp/easy_install-10gV93/flufl.enum-3.3.2/temp/tmpz01MIV/distribute-0.6.19
Building a Distribute egg in /tmp/easy_install-10gV93/flufl.enum-3.3.2
/tmp/easy_install-10gV93/flufl.enum-3.3.2/distribute-0.6.19-py2.7.egg
zip_safe flag not set; analyzing archive contents...
flufl.__init__: module references __path__
Got flufl.enum 3.3.2.
Getting distribution for 'flufl.bounce'.
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
Extracting in /tmp/easy_install-g21I2F/flufl.bounce-2.1.1/temp/tmpyEYnaw
Now working in /tmp/easy_install-g21I2F/flufl.bounce-2.1.1/temp/tmpyEYnaw/distribute-0.6.14
Building a Distribute egg in /tmp/easy_install-g21I2F/flufl.bounce-2.1.1
/tmp/easy_install-g21I2F/flufl.bounce-2.1.1/distribute-0.6.14-py2.7.egg
warning: no previously-included files found matching '*.egg'
no previously-included directories found matching 'build'
zip_safe flag not set; analyzing archive contents...
flufl.__init__: module references __path__
Got flufl.bounce 2.1.1.
Getting distribution for 'argparse'.
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
warning: no previously-included files matching '*.orig' found anywhere in distribution
warning: no previously-included files matching '*.rej' found anywhere in distribution
no previously-included directories found matching 'doc/_build'
no previously-included directories found matching 'env24'
no previously-included directories found matching 'env25'
no previously-included directories found matching 'env26'
no previously-included directories found matching 'env27'
zip_safe flag not set; analyzing archive contents...
Got argparse 1.2.1.
Getting distribution for 'zope.schema'.
warning: no files found matching '*.txt' under directory 'src/zope/schema'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Got zope.schema 4.2.0.
Getting distribution for 'zope.i18nmessageid'.
Got zope.i18nmessageid 4.0.0.
Getting distribution for 'WebOb'.
no previously-included directories found matching '*.pyc'
no previously-included directories found matching '*.pyo'
Got WebOb 1.2.1.
Getting distribution for 'mimeparse>=0.1.3'.
Got mimeparse 0.1.3.
Getting distribution for 'lazr.delegates'.
Got lazr.delegates 1.2.0.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/runner'.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/master'.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/mailman'.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/onebounce'.
Generated interpreter '/home/steve/src/Mailman3/mailman-trunk/bin/py'.
Installing tags.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/tags'.
Installing test.
Generated script '/home/steve/src/Mailman3/mailman-trunk/bin/test'.
steve@turnbull:~/src/Mailman3/mailman-trunk$ bin/test -vv
Running tests at level 1
Running mailman.testing.layers.ConfigLayer tests:
  Set up mailman.testing.layers.MockAndMonkeyLayer in 0.000 seconds.
  Set up mailman.testing.layers.ConfigLayer Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 380, in run_layer
    setup_layer(options, layer, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 672, in setup_layer
    layer.setUp()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/testing/layers.py", line 130, in setUp
    initialize.initialize_2(testing=True)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/initialize.py", line 153, in initialize_2
    config.db = getUtility(IDatabaseFactory, utility_name).create()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/factory.py", line 80, in create
    database.initialize()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/base.py", line 68, in initialize
    self._create(debug)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/base.py", line 141, in _create
    store = Store(database, GenerationalCache())
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/store.py", line 73, in __init__
    self._connection = database.connect(self._event)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/database.py", line 410, in connect
    return self.connection_factory(self, event)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/database.py", line 185, in __init__
    self._raw_connection = self._database.raw_connect()
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/databases/postgres.py", line 329, in raw_connect
    raw_connection = psycopg2.connect(self._dsn)
  File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
    connection_factory=connection_factory, async=async)
OperationalError: FATAL: password authentication failed for user "barry"
FATAL: password authentication failed for user "barry"

Running mailman.testing.layers.LMTPLayer tests:
  Set up mailman.testing.layers.ConfigLayer Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 380, in run_layer
    setup_layer(options, layer, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 667, in setup_layer
    setup_layer(options, base, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 672, in setup_layer
    layer.setUp()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/testing/layers.py", line 103, in setUp
    initialize.initialize_1(INHIBIT_CONFIG_FILE)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/initialize.py", line 125, in initialize_1
    mailman.config.config.load(config_path)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 101, in load
    self._post_process()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 120, in _post_process
    notify(ConfigurationUpdatedEvent(self))
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.event-4.0.0-py2.7.egg/zope/event/__init__.py", line 31, in notify
    subscriber(event)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/switchboard.py", line 268, in handle_ConfigurationUpdatedEvent
    'Duplicate runner name: {0}'.format(name))
AssertionError: Duplicate runner name: in

Running mailman.testing.layers.SMTPLayer tests:
  Set up mailman.testing.layers.ConfigLayer Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 380, in run_layer
    setup_layer(options, layer, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 667, in setup_layer
    setup_layer(options, base, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 672, in setup_layer
    layer.setUp()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/testing/layers.py", line 103, in setUp
    initialize.initialize_1(INHIBIT_CONFIG_FILE)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/initialize.py", line 125, in initialize_1
    mailman.config.config.load(config_path)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 101, in load
    self._post_process()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 120, in _post_process
    notify(ConfigurationUpdatedEvent(self))
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.event-4.0.0-py2.7.egg/zope/event/__init__.py", line 31, in notify
    subscriber(event)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/switchboard.py", line 268, in handle_ConfigurationUpdatedEvent
    'Duplicate runner name: {0}'.format(name))
AssertionError: Duplicate runner name: in

Running mailman.testing.layers.RESTLayer tests:
  Set up mailman.testing.layers.ConfigLayer Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 380, in run_layer
    setup_layer(options, layer, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 667, in setup_layer
    setup_layer(options, base, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 667, in setup_layer
    setup_layer(options, base, setup_layers)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.testrunner-4.0.4-py2.7.egg/zope/testrunner/runner.py", line 672, in setup_layer
    layer.setUp()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/testing/layers.py", line 103, in setUp
    initialize.initialize_1(INHIBIT_CONFIG_FILE)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/initialize.py", line 125, in initialize_1
    mailman.config.config.load(config_path)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 101, in load
    self._post_process()
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/config/config.py", line 120, in _post_process
    notify(ConfigurationUpdatedEvent(self))
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/zope.event-4.0.0-py2.7.egg/zope/event/__init__.py", line 31, in notify
    subscriber(event)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/switchboard.py", line 268, in handle_ConfigurationUpdatedEvent
    'Duplicate runner name: {0}'.format(name))
AssertionError: Duplicate runner name: in

Running zope.testrunner.layer.UnitTests tests:
  Tear down mailman.testing.layers.MockAndMonkeyLayer in 0.000 seconds.
  Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
  Running:
 test_acquire_lock_1 (mailman.bin.tests.test_master.TestMasterLock)
 test_master_state (mailman.bin.tests.test_master.TestMasterLock)
 test_current_working_directory (mailman.tests.test_configfile.TestConfigFileSearch)
 test_baseline (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_environment_variable (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_environment_variable_to_missing_path (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_etc_file (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_home_dot_file (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_sibling_directory (mailman.tests.test_configfile.TestConfigFileSearchWithChroot)
 test_adding_the_message_hash (mailman.utilities.tests.test_email.TestEmail)
 test_angle_brackets_dont_contribute_to_hash (mailman.utilities.tests.test_email.TestEmail)
 test_hash_header_left_alone_if_no_message_id (mailman.utilities.tests.test_email.TestEmail)
 test_mismatched_angle_brackets_do_contribute_to_hash (mailman.utilities.tests.test_email.TestEmail)
 test_no_at_split (mailman.utilities.tests.test_email.TestEmail)
 test_normal_split (mailman.utilities.tests.test_email.TestEmail)
 test_remove_hash_headers_first (mailman.utilities.tests.test_email.TestEmail)
 test_dont_honor_ws (mailman.utilities.tests.test_wrap.TestWrap)
 test_honor_ws (mailman.utilities.tests.test_wrap.TestWrap)
 test_indentation_boundary (mailman.utilities.tests.test_wrap.TestWrap)
 test_simple_wrap (mailman.utilities.tests.test_wrap.TestWrap)
 test_two_paragraphs (mailman.utilities.tests.test_wrap.TestWrap)
  Ran 21 tests with 0 failures and 0 errors in 0.020 seconds.
Tearing down left over layers:
  Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.

Tests with errors:
   Layer: mailman.testing.layers.ConfigLayer
   Layer: mailman.testing.layers.LMTPLayer
   Layer: mailman.testing.layers.SMTPLayer
   Layer: mailman.testing.layers.RESTLayer
Total: 21 tests, 0 failures, 4 errors in 0.464 seconds.
steve@turnbull:~/src/Mailman3/mailman-trunk$ python setup.py build_sphinx
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'build_sphinx'
steve@turnbull:~/src/Mailman3/mailman-trunk$ grep -r build_sphinx .
./eggs/passlib-1.6.1-py2.7.egg/passlib/_setup/docdist.py: self.run_command("build_sphinx")
./eggs/passlib-1.6.1-py2.7.egg/passlib/_setup/docdist.py: cmd = self.get_finalized_command("build_sphinx")
Binary file ./eggs/passlib-1.6.1-py2.7.egg/passlib/_setup/docdist.pyo matches
Binary file ./eggs/passlib-1.6.1-py2.7.egg/passlib/_setup/docdist.pyc matches
./src/mailman/docs/START.rst: % python setup.py build_sphinx
./src/mailman/__init__.py:if 'build_sphinx' not in sys.argv:
Binary file ./src/mailman/__init__.pyc matches
steve@turnbull:~/src/Mailman3/mailman-trunk$ mkdir instance; cd instance
steve@turnbull:~/src/Mailman3/mailman-trunk$ xemacs mailman.cfg
steve@turnbull:~/src/Mailman3/mailman-trunk/instance$ ../bin/mailman start
Starting Mailman's master runner
steve@turnbull:~/src/Mailman3/mailman-trunk/instance$ ../bin/mailman create -d -o <email address hidden> <email address hidden>
Created mailing list: <email address hidden>
steve@turnbull:~/src/Mailman3/mailman-trunk/instance$ ../bin/mailman info -vGNU Mailman 3.0.0b1+ (Freeze)
Python 2.7.3rc2 (default, Apr 22 2012, 22:30:17)
[GCC 4.6.3]
config file: /home/steve/src/Mailman3/mailman-trunk/instance/mailman.cfg
db url: sqlite:////home/steve/src/Mailman3/mailman-trunk/instance/var/data/mailman.db
REST root url: http://localhost:8001/3.0/
REST credentials: restadmin:restpass
File system paths:
    ARCHIVE_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/archives
    BIN_DIR = /home/steve/src/Mailman3/mailman-trunk/bin
    DATA_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/data
    ETC_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/etc
    EXT_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/ext
    LIST_DATA_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/lists
    LOCK_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/locks
    LOCK_FILE = /home/steve/src/Mailman3/mailman-trunk/instance/var/locks/master.lck
    LOG_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/logs
    MESSAGES_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/messages
    PID_FILE = /home/steve/src/Mailman3/mailman-trunk/instance/var/master.pid
    QUEUE_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/queue
    TEMPLATE_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var/templates
    VAR_DIR = /home/steve/src/Mailman3/mailman-trunk/instance/var

######################## instance/var/logs/mailman.log ########################
Aug 08 23:56:45 2012 (16782) command failure: /usr/sbin/postmap /home/steve/src/Mailman3/mailman-trunk/instance/var/data/postfix_lmtp, 127, Key has expired
Aug 09 12:15:33 2012 (7571) DigestRunner runner started.
Aug 09 12:15:33 2012 (7570) IncomingRunner runner started.
Aug 09 12:15:33 2012 (7579) LMTPRunner runner started.
Aug 09 12:15:33 2012 (7574) OutgoingRunner runner started.
Aug 09 12:15:33 2012 (7575) VirginRunner runner started.
Aug 09 12:15:33 2012 (7580) CommandRunner runner started.
Aug 09 12:15:33 2012 (7576) RetryRunner runner started.
Aug 09 12:15:33 2012 (7581) NNTPRunner runner started.
Aug 09 12:15:34 2012 (7577) RESTRunner runner started.
Aug 09 12:15:34 2012 (7577) Starting REST server
Aug 09 12:15:34 2012 (7573) ArchiveRunner runner started.
Aug 09 12:15:34 2012 (7572) PipelineRunner runner started.
Aug 09 12:15:35 2012 (7578) BounceRunner runner started.
Aug 09 13:39:41 2012 (7570) Uncaught runner exception: Expected unicode, found <type 'datetime.datetime'>: datetime.datetime(2012, 8, 9, 4, 39, 41, 475922)
Aug 09 13:39:41 2012 (7570) Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/runner.py", line 138, in _one_iteration
    self._process_one_file(msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/runner.py", line 221, in _process_one_file
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/runners/incoming.py", line 67, in _dispose
    process(mlist, msg, msgdata, start_chain)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/chains.py", line 90, in process
    link.function(mlist, msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/chains/hold.py", line 142, in _process
    request_id = hold_message(mlist, msg, msgdata, None)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/app/moderator.py", line 105, in hold_message
    RequestType.held_message, message_id, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/transaction.py", line 78, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/requests.py", line 91, in hold_request
    token = getUtility(IPendings).add(pendable, timedelta(days=5000))
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/transaction.py", line 78, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/pending.py", line 131, in add
    keyval = PendedKeyValue(key=key, value=value)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/pending.py", line 54, in __init__
    self.value = value
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/properties.py", line 67, in __set__
    obj_info.variables[column].set(value)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/variables.py", line 396, in parse_set
    % (type(value), value))
TypeError: Expected unicode, found <type 'datetime.datetime'>: datetime.datetime(2012, 8, 9, 4, 39, 41, 475922)
Aug 09 13:39:41 2012 (7570) SHUNTING: 1344487181.861564+93e44eec2008ac77dfaf0df4676c6b5ef2b68de8
Aug 09 13:42:09 2012 (7570) Uncaught runner exception: Expected unicode, found <type 'datetime.datetime'>: datetime.datetime(2012, 8, 9, 4, 42, 8, 769294)
Aug 09 13:42:09 2012 (7570) Traceback (most recent call last):
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/runner.py", line 138, in _one_iteration
    self._process_one_file(msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/runner.py", line 221, in _process_one_file
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/runners/incoming.py", line 67, in _dispose
    process(mlist, msg, msgdata, start_chain)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/core/chains.py", line 90, in process
    link.function(mlist, msg, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/chains/hold.py", line 142, in _process
    request_id = hold_message(mlist, msg, msgdata, None)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/app/moderator.py", line 105, in hold_message
    RequestType.held_message, message_id, msgdata)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/transaction.py", line 78, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/requests.py", line 91, in hold_request
    token = getUtility(IPendings).add(pendable, timedelta(days=5000))
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/database/transaction.py", line 78, in wrapper
    return function(args[0], config.db.store, *args[1:], **kws)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/pending.py", line 131, in add
    keyval = PendedKeyValue(key=key, value=value)
  File "/home/steve/src/Mailman3/mailman-trunk/src/mailman/model/pending.py", line 54, in __init__
    self.value = value
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/properties.py", line 67, in __set__
    obj_info.variables[column].set(value)
  File "/home/steve/src/Mailman3/mailman-trunk/eggs/storm-0.19-py2.7-linux-x86_64.egg/storm/variables.py", line 396, in parse_set
    % (type(value), value))
TypeError: Expected unicode, found <type 'datetime.datetime'>: datetime.datetime(2012, 8, 9, 4, 42, 8, 769294)
Aug 09 13:42:09 2012 (7570) SHUNTING: 1344487329.078011+1a488523540df1b454f0797dcae59dfd3dac27b9

######################## instance/mailman.cfg ########################

# Copyright (C) 2012 by the Free Software Foundation, Inc.
#
# This file was contributed to GNU Mailman as an example
#
# This is an example local configuration file for an Exim installation.

[mailman]

layout: dev

[devmode]
enabled: yes
recipient: <email address hidden>

[language.en]

charset: utf-8

[mta]

# The class defining the interface to the incoming mail transport agent.
incoming: mailman.mta.exim.LMTP

# The callable implementing delivery to the outgoing mail transport agent.
# This must accept three arguments, the mailing list, the message, and the
# message metadata dictionary.
outgoing: mailman.mta.deliver.deliver

# The runners are defined in the master "mailman.cfg".

# The style is defined in the master "mailman.cfg".

######################## src/mailman/src/mta/exim.py ########################

# N.B. All of the methods of class LMTP can probably be replaced with
# trivial methods that just "pass". However I haven't tested that
# yet, so I include the exim.py that was actually used when I created
# my test list "<email address hidden>".

# Copyright (C) 2001-2012 by the Free Software Foundation, Inc.
#
# This file is part of GNU Mailman.
#
# GNU Mailman is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation, either version 3 of the License, or (at your option)
# any later version.
#
# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
# more details.
#
# You should have received a copy of the GNU General Public License along with
# GNU Mailman. If not, see <http://www.gnu.org/licenses/>.

"""Creation/deletion hooks for the Exim MTA.

This file has been used with and lightly tested on a Debian system
with the exim4-4.80-4 package.

#### Move this to user-accessible documentation:
Exim doesn't use an alias file.
Instead, the address is filtered by the Exim router and passed to a transport.
Both the relevant router and the transport are named 'mailman3'.
The transport is just an smtp driver with lmtp protocol.
The router needs to accept any name, and parse out the list name from the
administrative aliases. In order to check whether string is actually a
list name, we have Exim check for the directory $var_dir/lists with the same
name as the list's posting address.
AFAICS, creating lists is a no-op here.
I use main/15_local_mailman_macros, router/444_local_mailman3, and
transport/54_local_mailman3 on a Debian system.
"""

from __future__ import absolute_import, print_function, unicode_literals

__metaclass__ = type
__all__ = [
    'LMTP',
    ]

# #### Most of these imports can go away, I suppose
import os
import logging

from flufl.lock import Lock
from operator import attrgetter
from zope.component import getUtility
from zope.interface import implementer

from mailman.config import config
from mailman.interfaces.listmanager import IListManager
from mailman.interfaces.mta import (
    IMailTransportAgentAliases, IMailTransportAgentLifecycle)
from mailman.utilities.datetime import now

# #### We don't need this AFAICS.
# log = logging.getLogger('mailman.error')

# #### This can be removed, but that is untested.
def _receipt(mlist):
    return os.path.join(config.VAR_DIR, 'receipts', mlist.list_name)

@implementer(IMailTransportAgentLifecycle)
class LMTP:
    """Connect Mailman to Exim via LMTP."""

    # #### This probably can be replaced with "def create(s, m): pass",
    # but that is untested.

    def create(self, mlist):
        """See `IMailTransportAgentLifecycle`.

        Exim detects lists in its router. We provide a dummy entry in
        the file system in the 'receipts' directory for Exim's check."""

        # Acquire a lock file to prevent other processes from racing us here.
        lock_file = os.path.join(config.LOCK_DIR, 'mta')
        with Lock(lock_file):
            os.makedirs(_receipt(mlist))

    # #### This probably can be replaced with "def delete(s, m): pass",
    # but that is untested.

    def delete(self, mlist):
        """See `IMailTransportAgentLifecycle`.

        Removes mlist's receipt."""

        # Acquire a lock file to prevent other processes from racing us here.
        lock_file = os.path.join(config.LOCK_DIR, 'mta')
        with Lock(lock_file):
            os.rmdir(_receipt(mlist))

    def regenerate(self, output=None):
        """See `IMailTransportAgentLifecycle`.

        This routine does nothing.
        """
        pass

######################## Exim macros for Mailman3 ########################
domainlist mm_domains=turnbull.sk.tsukuba.ac.jp

MM3_LMTP_PORT=8024
MM3_HOME=/home/steve/src/Mailman3/mailman-trunk/instance
# #### Next two unused?
MM3_UID=list
MM3_GID=list

# The path of the Mailman mail wrapper script
MM3_WRAP=MM3_HOME/bin/mailman
# The path of the list receipt (only used as a required file when
# verifying list addresses)
MM3_LISTCHK=MM3_HOME/var/lists/${local_part}@${domain}

######################## Exim router for Mailman3 ########################

# Routers are the only configuration that is order dependent. In my
# Debian configuration, the following routers precede Mailman3:
#
# 00_exim4-config_header
# 100_exim4-config_domain_literal
# 150_exim4-config_hubbed_hosts
# 200_exim4-config_primary
# 300_exim4-config_real_local
# 400_exim4-config_system_aliases
# 430_ack_class_mail # a local hack
# 440_class_mail # a local hack (cont.)
#
# It should be reasonably clear what these do but I'll provide details
# on request.

mailman3_router:
  driver = accept
  domains = +mm_domains
  require_files = MM3_LISTCHK
  local_part_suffix_optional
  local_part_suffix = -admin : \
         -bounces : -bounces+* : \
         -confirm : -confirm+* : \
         -join : -leave : \
         -owner : -request : \
         -subscribe : -unsubscribe
  transport = mailman3_transport

######################## Exim transport for Mailman3 ########################

mailman3_transport:
  driver = smtp
  protocol = lmtp
  allow_localhost
  hosts = localhost
  port = MM3_LMTP_PORT

Tags: mailman3
Revision history for this message
Stephen Turnbull (stephen-xemacs) wrote :

This is probably not Bug #949924, which should have been fixed a few commits back.

Revision history for this message
Stephen Turnbull (stephen-xemacs) wrote :

The following patch works around, but I don't think it's right. I suspect that something imported "datetime" where it should have imported mailman.utils.datetime or something like that.

--- src/mailman/model/pending.py 2012-04-26 02:08:22 +0000
+++ src/mailman/model/pending.py 2012-08-13 07:38:39 +0000
@@ -26,6 +26,7 @@
     ]

+import datetime
 import time
 import random
 import hashlib
@@ -128,6 +129,9 @@
                 # We expect this to be a list of strings.
                 value = ('mailman.model.pending.unpack_list\1' +
                          '\2'.join(value))
+ # #### Evile hack!!
+ elif type(value) is datetime.datetime:
+ value = unicode(value)
             keyval = PendedKeyValue(key=key, value=value)
             pending.key_values.add(keyval)
         store.add(pending)

Revision history for this message
Stephen Turnbull (stephen-xemacs) wrote :

I forgot to mention that with this patch my test posts were unshunted and made it to the moderation queue. What happened from there is a new bug.

tags: added: mailman3
Revision history for this message
Barry Warsaw (barry) wrote :

I'll also note that the test failure is because I accidentally left the postgres configuration in testing.cfg. It's my own crufty local postgres installation. I've fixed that since then by restoring the default sqlite data for the tests.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.