Postgresql support is broken

[Abhilash Raj]

The support for postgresql database backend is broken (I guess after the python3 port). The details of the failures can be found here with the continuous integration on gitlab https://ci.gitlab.com/projects/828

[Barry Warsaw (barry)]

Ug. Lots of pg failures :(

[Barry Warsaw (barry)]

Try running the following:

$ MAILMAN_EXTRA_TESTING_CFG=/home/barry/projects/mailman/postgres.cfg tox -e pg

where you replace the path to the postgres.cfg with one of your own as described in src/mailman/docs/START.rst

Pretty quickly we hit an exception:

======================================================================
ERROR: LayerSuite
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: column "id" does not exist
LINE 1: ...LECT setval('"domain_owner_id_seq"', coalesce(max("id"), 1),
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/barry/projects/mailman/trunk/src/mailman/testing/layers.py", line 207, in testTearDown
    reset_the_world()
  File "/home/barry/projects/mailman/trunk/src/mailman/testing/helpers.py", line 476, in reset_the_world
    config.db._reset()
  File "/home/barry/projects/mailman/trunk/src/mailman/database/factory.py", line 123, in _reset
    self._post_reset(self.store)
  File "/home/barry/projects/mailman/trunk/src/mailman/database/postgresql.py", line 49, in _post_reset
    """.format(table))
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 1023, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1332, in _handle_dbapi_exception
    exc_info
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 188, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 181, in reraise
    raise value.with_traceback(tb)
  File "/home/barry/projects/mailman/trunk/.tox/pg/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/home/barry/projects/...

[Aurélien Bompard (abompard)]

Alright I just linked to a branch with 4 commits fixing all the PostgreSQL issues. Please review, I may have left print statements in the code... ;-)

[Barry Warsaw (barry)]

Thanks! Looking over the diff I think I understand all the changes. I'm running the test suite now for both pg and sqlite.

The one change I don't understand is to the importer code. Why did you choose to do it this way instead of the old way?