ubuntu 24.04 mailman3-web fails install on clean system

[Thomas]

hello y'all.

as of now mailman3-web fails to install on noble with the following error:

Creating config file /etc/dbconfig-common/mailman3-web.conf with new version
creating database mailman3web.db: success.
verifying database mailman3web.db exists: success.

Creating config file /etc/mailman3/mailman-web.py with new version
/usr/lib/python3/dist-packages/django_q/conf.py:174: UserWarning: Retry and timeout are misconfigured. Set retry larger than timeout,failure to do so will cause the tasks to be retriggered before completion.See https://django-q2.readthedocs.io/en/master/configure.html#retry for details.
  warn(
Traceback (most recent call last):
  File "/usr/bin/django-admin", line 33, in <module>
    sys.exit(load_entry_point('Django==4.2.11', 'console_scripts', 'django-admin')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/usr/lib/python3/dist-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib/python3/dist-packages/django/apps/registry.py", line 124, in populate
    app_config.ready()
  File "/usr/lib/python3/dist-packages/allauth/account/apps.py", line 15, in ready
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: allauth.account.middleware.AccountMiddleware must be added to settings.MIDDLEWARE
dpkg: error processing package mailman3-web (--configure):
 installed mailman3-web package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 mailman3-web
needrestart is being skipped since dpkg has failed
E: Sub-process /usr/bin/dpkg returned an error code (1)

I guess a fresh install should not fail like that.

cheers!

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Marco van Zwetselaar (zwets)]

Can confirm this also occurs when upgrading from mantic to noble.

The issue appears to be caused by a breaking change introduced in version 1.3.10 (2023-10-21)[1], which is the version of python3-django-postorius in noble.

Adding "allauth.account.middleware.AccountMiddleware" to the MIDDLEWARE list in /usr/share/mailman3/settings.py fixes the issue, but I then run into other incompatibility errors. First there is this:

   File "/opt/mailman/mm/venv_rel/lib/python3.9/site-packages/hyperkitty/lib/renderer.py", line 6, in <module>
        from mistune.plugins.extra import plugin_url
   ModuleNotFoundError: No module named 'mistune.plugins.extra'

which appears to be an upstream issue[2] that is harder to solve on our end because it requires a downgrade to of python-mistune to 2.x, whereas noble ships with 3.0.2.[*]

I briefly tested with the referencing code (in /usr/lib/python3/dist-packages/hyperkitty/lib/renderer.py) patched out. The mailman3 web application then does come up, but now spews periodic errors to the log that suggest that a database migration should have run:

    10:11:10 [Q] ERROR Could not create task from schedule
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 89, in _execute
        return self.cursor.execute(sql, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    psycopg2.errors.UndefinedColumn: column django_q_schedule.intended_date_kwarg does not exist
    LINE 1: ..._schedule"."task", "django_q_schedule"."cluster", "django_q_...

I would guess this issue stems from the upgrade of python3-django-q to 1.6.1 in noble. I did not check if upstream provides a migration script that did not make it into the deb package, or have stuck with 1.3.9 (in which case again it will be harder to solve on our end, because noble comes with 1.6.1)[*].

Here is where I stopped chasing this down, but there may be light at the end of the tunnel: the Debian maintainer yesterday committed a snapshot of the latest upstream[3], which is supposed to fix the allauth issue[4] as well as several others.

For now, I'll try rolling back to the mantic packages, which worked great (with patches for #2060293, #2060276, and #2060753).

[*] On a more general note: wouldn't it be much simpler and less brittle to do away with system-wide python3-django-* packages, and instead copy their source code into the mailman3-web package?

I'm all for the parsimony of shared libraries (in fact, snap and flatpak make me cringe) but the python-django-* debs are tiny, source-only modules intended as library code for Python web-applications. Why not reuse them by copying into the source tree? They're minuscule and evolve rapidly, with little regard for backward compatibility. Exactly what you wouldn't want as a (quasi-)binary system-wide library.

Cheers
Marco

[1] https://pypi.org/project/django-mailman3/
[2] https://gitlab.com/mailman/hyperkitty/-/issues/437
[3] https://salsa.debian.org/mailman-team/mailman-suite/-/commit/edf7873dce29e102c03a2be01133a6c00f8994c0
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1062707.