fail2ban is broken in 24.04 Noble

[Robert Dinse]

[ Impact ]
* fail2ban fails to start on Ubuntu 24.04, due to new Python and deprecated classes (now vendored upstream)

[ Test Plan ]
* Install and check systemctl status fail2ban.

[ Where problems could occur ]
* Nowhere, fixing a program failing to start

[ Other info ]
Was working in 22.04, but after upgrading to 24.04 I get this:

× fail2ban.service - Fail2Ban Service
     Loaded: loaded (/etc/systemd/system/fail2ban.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Sun 2024-02-25 02:27:29 PST; 1 day 18h ago
   Duration: 2.330s
       Docs: man:fail2ban(1)
    Process: 1130 ExecStartPre=/bin/mkdir -p /run/fail2ban (code=exited, status=0/SUCCESS)
    Process: 1133 ExecStart=/usr/local/bin/fail2ban-server -xf start (code=exited, status=255/EXCEPTION)
   Main PID: 1133 (code=exited, status=255/EXCEPTION)
        CPU: 660ms

Feb 25 02:27:26 mx1 systemd[1]: Starting fail2ban.service - Fail2Ban Service...
Feb 25 02:27:26 mx1 systemd[1]: Started fail2ban.service - Fail2Ban Service.
Feb 25 02:27:28 mx1 fail2ban-server[1133]: 2024-02-25 02:27:28,952 fail2ban [1133]: ERROR No module named 'asynchat'
Feb 25 02:27:29 mx1 systemd[1]: fail2ban.service: Main process exited, code=exited, status=255/EXCEPTION
Feb 25 02:27:29 mx1 systemd[1]: fail2ban.service: Failed with result 'exit-code'.

grep fail2ban syslog
2024-02-25T02:25:17.813593-08:00 mx1 systemd[1]: Stopping fail2ban.service - Fail2Ban Service...
2024-02-25T02:27:26.625640-08:00 mx1 systemd[1]: Starting fail2ban.service - Fail2Ban Service...
2024-02-25T02:27:26.678572-08:00 mx1 systemd[1]: Started fail2ban.service - Fail2Ban Service.
2024-02-25T02:27:28.954548-08:00 mx1 fail2ban-server[1133]: 2024-02-25 02:27:28,952 fail2ban [1133]: ERROR No module named 'asynchat'
2024-02-25T02:27:29.004733-08:00 mx1 systemd[1]: fail2ban.service: Main process exited, code=exited, status=255/EXCEPTION
2024-02-25T02:27:29.004834-08:00 mx1 systemd[1]: fail2ban.service: Failed with result 'exit-code'.

Seems 24.04 is missing a python 3.12 module 'asynchat'.

This leaves my systems vulnerable to brute-force password guessing attacks.

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: fail2ban 1.0.2-3
Uname: Linux 6.7.6 x86_64
ApportVersion: 2.28.0-0ubuntu1
Architecture: amd64
CasperMD5CheckResult: unknown
CurrentDesktop: MATE
Date: Mon Feb 26 20:33:12 2024
InstallationDate: Installed on 2017-08-14 (2388 days ago)
InstallationMedia: Ubuntu-MATE 17.04 "Zesty Zapus" - Release amd64 (20170412)
PackageArchitecture: all
SourcePackage: fail2ban
UpgradeStatus: Upgraded to noble on 2024-01-23 (35 days ago)

[Alex Murray (alexmurray)]

asynchat was removed in python 3.12, which just became the default python3 in 24.04

[Alex Murray (alexmurray)]

Relevant upstream issue https://github.com/fail2ban/fail2ban/issues/3487

[Alex Murray (alexmurray)]

So whilst in Ubuntu we do have python-pyasyncore which provides asyncore, we don't have asynchat so this might need to be packaged separately OR vendored into fail2ban

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robert Dinse (nanook)]

I wish people wouldn't use languages like python which changes it's packages and syntax weekly, but since they do the obvious question, is anyone going to work on this? It is a rather important security tool.

[Åka Sikrom (akrosikam)]

As of this commit from 12 Dec 2023: https://github.com/fail2ban/fail2ban/commit/1024452fe1befeb5a0a014386a81ec183cd45bb5
....upstream ships its source with the missing async* libraries embedded, which solves the issue for now.

Here is a suggested workaround for Noble until the fix hopefully finds its way into the repos.

DISCLAMER: Try at your own risk. I am not an Ubuntu/Debian developer. Next time you receive a proper package update of fail2ban via apt, any of the manually-downloaded files below may cause conflicts and break your system.

mkdir -m 0755 /usr/lib/python3/dist-packages/fail2ban/compat
wget -O /usr/lib/python3/dist-packages/fail2ban/compat/asynchat.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/compat/asynchat.py
wget -O /usr/lib/python3/dist-packages/fail2ban/compat/asyncore.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/compat/asyncore.py
cp -p /usr/lib/python3/dist-packages/fail2ban/server/asyncserver.py{,.original}
wget -O /usr/lib/python3/dist-packages/fail2ban/server/asyncserver.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/server/asyncserver.py
chmod 0644 /usr/lib/python3/dist-packages/fail2ban/{compat/async{hat,ore}.py,server/asyncserver.py}

The fail2ban package does not have 'python3-setuptools' listed under "Depends" in its control file, although it requires distutils (provided by the setuptools package) for its default sshd jail and thus for fail2ban to be able to run at all. So, unless you have already installed it, you will need to install it now:

apt install python3-setuptools

Once done, fail2ban should start successfully.

[Robert Dinse (nanook)]

      This does make it run. Thank you.

-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-
  Eskimo North Linux Friendly Internet Access, Shell Accounts, and Hosting.
    Knowledgeable human assistance, not telephone trees or script readers.
  See our web site: http://www.eskimo.com/ [phone] or [phone].

On Sun, 10 Mar 2024, Åka Sikrom wrote:

> Date: Sun, 10 Mar 2024 12:30:18 -0000
> From: Åka Sikrom <email address hidden>
> To: <email address hidden>
> Subject: [Bug 2055114] Re: fail2ban is broken in 24.04 Noble
>
> As of this commit from 12 Dec 2023: https://github.com/fail2ban/fail2ban/commit/1024452fe1befeb5a0a014386a81ec183cd45bb5
> ....upstream ships its source with the missing async* libraries embedded, which solves the issue for now.
>
>
> Here is a suggested workaround for Noble until the fix hopefully finds its way into the repos.
>
> DISCLAMER: Try at your own risk. I am not an Ubuntu/Debian developer.
> Next time you receive a proper package update of fail2ban via apt, any
> of the manually-downloaded files below may cause conflicts and break
> your system.
>
> mkdir -m 0755 /usr/lib/python3/dist-packages/fail2ban/compat
> wget -O /usr/lib/python3/dist-packages/fail2ban/compat/asynchat.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/compat/asynchat.py
> wget -O /usr/lib/python3/dist-packages/fail2ban/compat/asyncore.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/compat/asyncore.py
> cp -p /usr/lib/python3/dist-packages/fail2ban/server/asyncserver.py{,.original}
> wget -O /usr/lib/python3/dist-packages/fail2ban/server/asyncserver.py https://github.com/fail2ban/fail2ban/raw/1024452fe1befeb5a0a014386a81ec183cd45bb5/fail2ban/server/asyncserver.py
>
> My custom-boostrapped Noble test system is also missing the required
> 'distutils' Python module by default. The fail2ban package does not have
> 'python3-setuptools' listed under "Depends" in its control file, which
> it possibly should at this point. Official Ubuntu images may already
> have it installed via other packages, but if not, you will need to
> install it as well:
>
> apt install python3-setuptools
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2055114
>
> Title:
> fail2ban is broken in 24.04 Noble
>
> Status in fail2ban package in Ubuntu:
> Confirmed
>
> Bug description:
> Was working in 22.04, but after upgrading to 24.04 I get this:
>
> × fail2ban.service - Fail2Ban Service
> Loaded: loaded (/etc/systemd/system/fail2ban.service; enabled; preset: enabled)
> Active: failed (Result: exit-code) since Sun 2024-02-25 02:27:29 PST; 1 day 18h ago
> Duration: 2.330s
> Docs: man:fail2ban(1)
> Process: 1130 ExecStartPre=/bin/mkdir -p /run/fail2ban (code=exited, status=0/SUCCESS)
> Process: 1133 ExecStart=/usr/local/bin/fail2ban-server -xf start (code=exited, status=255/EXCEPTION)
> Main PID: 1133 (code=exited, status=255/EXCEPTION)
> CPU: 660ms
>
> Feb 25 02:27:26 mx1 systemd[1]: Starting fail2ban.service - Fail2Ban...

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "fail2ban-noble-add-bundled-async-python3.12.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Åka Sikrom (akrosikam)]

Maintainers: please find attached a new version of my suggested source patch.

Changes:
* Bundle asyncore.py, asynchat.py and updated asyncserver.py (manual cherry-pick of upstream commit 1024452)
* Add 'python3-setuptools' to Depends

I have deliberately *not* inserted anything into the changelog as I believe this should all be checked and signed off by somebody who knows what they are doing.

Regarding Debian, trixie still ships python3.11 as of today, so this issue does not apply to them yet.

Thanks.

[Adam Williamson (awilliamson)]

FWIW, this is what we did in Fedora: https://src.fedoraproject.org/rpms/fail2ban/c/24c973f252f6d1b4231ee49ee4d5efac785a2fe8?branch=rawhide

i.e., we depend on packages of the asynchat and asyncore libs that were added to pypi for this kind of purpose - https://pypi.org/project/pyasynchat/ and https://pypi.org/project/pyasyncore/ - and we also had to disable a couple of tests. I don't know if Ubuntu/Debian packaged the split-out asynchat and asyncore libs yet.

[Grant Slater (firefishy)]

Upstream have release a new version which fixes Python 3.12+ compatibility. https://github.com/fail2ban/fail2ban/releases/tag/1.1.0

[fossfreedom (fossfreedom)]

I see a patch has been added - so I've subscribed ubuntu sponsors.

However, please can the following template be used to describe the patch, how to test, understand the risks etc. https://wiki.ubuntu.com/SponsorshipProcess

[Åka Sikrom (akrosikam)]

Thanks fossfreedom,

I have tried to go through the documentation you linked to, but its links to specific processes are either dead or redirect to a new WIP readthedocs page where the info I need has not been addded yet. Since this would be my first-ever Ubuntu patch, I need a bit more specific information in order to improve my process and patch content.

If there is anything specific missing in my patch and/or its description above, please feel free to point it out, and I will do what I can.

Again, I have deliberately refrained from including a changelog entry / version bump, and the patch is not in quilt format since it is unclear to me whether it should be. If these are the only missing bits, I would be happy to add them and upload a revised patch.

[Sylvestre Ledru (sylvestre)]

I uploaded the new version in Debian:
https://tracker.debian.org/news/1524869/accepted-fail2ban-110-1-source-into-unstable/
if someone wants to sync it into Ubuntu, most welcome :)

[Andreas Hasenack (ahasenack)]

New version is in oracular-proposed[1]:

fail2ban (1.1.0-1) unstable; urgency=medium

  * New upstream release
    (LP: #2055114)
  * Block ssh invalid keys too (Closes: #1038779)
  * Follow upstream advice
    https://github.com/fail2ban/fail2ban/issues/3292#issuecomment-2078361360
    to only have sshd as enabled = true in jail.d_defaults-debian.conf
  * Update lintian override info format in d/source/lintian-overrides on line 1-2.
  * Update standards version to 4.6.2, no changes needed.

 -- Sylvestre Ledru <email address hidden> Thu, 02 May 2024 13:57:06 +0200

Nothing needs to be sponsored for the ubuntu devel release now. I'll mark it as fix committed, since it was uploaded.

Noble is still affected, so I'm adding a noble task. This then needs to become an SRU, following [2], before anything can be uploaded to noble.

1. https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1
2. https://wiki.ubuntu.com/StableReleaseUpdates

[fprietog (fprietog)]

@ahasenack thank you very much.

In the meantime as a workaround for noble you can install the oracular-proposed deb file "fail2ban_1.1.0-1_all.deb".

[Anthony MAUGENDRE (hac76)]

@fprietog workaround works like a charm ! thank you

wget https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1/+build/28291332/+files/fail2ban_1.1.0-1_all.deb
sudo dpkg -i fail2ban_1.1.0-1_all.deb

[Launchpad Janitor (janitor)]

This bug was fixed in the package fail2ban - 1.1.0-1

---------------
fail2ban (1.1.0-1) unstable; urgency=medium

  * New upstream release
    (LP: #2055114)
  * Block ssh invalid keys too (Closes: #1038779)
  * Follow upstream advice
    https://github.com/fail2ban/fail2ban/issues/3292#issuecomment-2078361360
    to only have sshd as enabled = true in jail.d_defaults-debian.conf
  * Update lintian override info format in d/source/lintian-overrides on line 1-2.
  * Update standards version to 4.6.2, no changes needed.

 -- Sylvestre Ledru <email address hidden> Thu, 02 May 2024 13:57:06 +0200

[Graham Inggs (ginggs)]

Marking the Noble task incomplete and unsubscribing ubuntu-sponsors for now.

[Immanuel Raj (iamimmanuelraj)]

How long till a fix is relreased? Its kinda very important to ubuntu server system administrator like me. Please do the needful as soon as possible

[Daniel Leidert (dleidert-deactivatedaccount)]

Can this *please* finally be fixed asap? This has been known for more than two months now, and even before the official release of Ubuntu 24.04 LTS had been made. How this issue is handled for the LTS release is becoming quite a source of frustration.

[Grant Slater (firefishy)]

Do we have to subscribe to Ubuntu Pro to get an update of this package? </sarcasm>

But seriously, what will it take to get this package fixed in Ubuntu? How can we take Ubuntu seriously on the server if such a fundamental package remains critically broken?

[Sylvestre Ledru (sylvestre)]

I don't know well the Ubuntu processes but I opened this sync request:
https://bugs.launchpad.net/bugs/2065796

[Andros (r-andros)]

They should really adopt version 1.1.0. Anything else makes no sense.

[fredbcode (fredbcode)]

Sylvestre Ledru (sylvestre) for mantic only, not noble ?

[Niv Shtibel (nivsv)]

having the same issue how do I make it work on Noble version of ubuntu please help

[Chris Cowley (chris-cowley)]

Fortunately for me, I only have to manage a single server and I patched this problem on that server myself (the comments from Aka Sikrom here provide some very useful pointers in this regard) so this no longer matters to me personally. But it's getting embarrassing how long this simple fix is taking to get to Noble.

There is clearly something very wrong with the process; I am reminded of Douglas Adams' quote about trying to grill a steak by having a succession of people coming into the room and breathing on it.

[Speculos (speculos)]

Hi,

As a temporary workaround I have personally switch to a docker container to do the job.

Ofc you must have docker installed and it is not the best option but it works like a charm for me.

https://docs.linuxserver.io/images/docker-fail2ban/

Waiting as all of you to see a proper fix be proposed.

See ya.

[snowieoxs (snowieoxs)]

Okay, I have been looking at all the comments, and provided links in this thread for a couple hours now, and I'm still not sure what needs to be done here. I do see there are workarounds, but what does a guy need to do to make it so that the apt package works with noble? This is a serious security issue that needs attention, If anyone wants try and get this to working this week I am available to help test/collab.

[Gianfranco Costamagna (costamagnagianfranco)]

Hello, the patch mentioned in https://bugs.launchpad.net/ubuntu/+source/fail2ban/+bug/2055114/comments/14
looks adding an additional dependency, without explaining why it was added.
Can you please provide more information and also a changelog? It can be changed later, but a schema of changelog is useful anyway for the sponsor.

[Åka Sikrom (akrosikam)]

Changelog added to patch, and patch hereby re-uploaded.

* Bundle required async* modules for python3.12 (Fixes: #2055114)
   see upstream commit https://github.com/fail2ban/fail2ban/commit/1024452
* debian/control: Add python3-setutools to Depends
   to provide required module 'distutils' (Fixes: #2055114)

[Gianfranco Costamagna (costamagnagianfranco)]

Why, add it as runtime dependency? setuptools is already a build dependency from what I can see...

[Åka Sikrom (akrosikam)]

Because it provides 'distutils' which, as it stands, is a required runtime module.

[Gianfranco Costamagna (costamagnagianfranco)]

Uploaded to Ubuntu without the additional dependency

[Åka Sikrom (akrosikam)]

Then I assume you have tested and verified (with a build of the package which you say you have now uploaded) that fail2ban.service starts successfully post-install, while python3-setuptools is *not* installed on your system?

[Gianfranco Costamagna (costamagnagianfranco)]

I uploaded on my ppa
https://launchpad.net/~costamagnagianfranco/+archive/ubuntu/costamagnagianfranco-ppa/+packages
please have a look!

[sparks71 (sparks71)]

I have installed the oracular version (1.1.0-2) on my 3 servers.
Why should I downgrade? failed udate policy

As I know ubuntu LTS, the package will not get any updates for the next 10 years anyway, so I can also install it via dpkg - makes no difference.

[Åka Sikrom (akrosikam)]

@Gianfranco: Done. As expected, fail2ban.service fails to start post-install, since you excluded an essential part of my patch. See attached terminal log output.

[Chris Cowley (chris-cowley)]

Have just tried this myself in a freshly-created Noble install. I can confirm Åka Sikrom (akrosikam) is absolutely correct; Python3-setuptools is a runtime dependency of fail2ban1.0.2-3ubuntu

[Gianfranco Costamagna (costamagnagianfranco)]

Ok, so if this is a runtime dependency, does this mean oracular is not yet fixed?

fail2ban_1.1.0-2_all.deb
------------------------

 new Debian package, version 2.0.
 size 465720 bytes: control archive=10096 bytes.
    7008 bytes, 175 lines conffiles
    1581 bytes, 33 lines control
   24895 bytes, 252 lines md5sums
    3418 bytes, 103 lines * postinst #!/bin/sh
    1820 bytes, 64 lines * postrm #!/bin/sh
     233 bytes, 7 lines * preinst #!/bin/sh
     832 bytes, 22 lines * prerm #!/bin/sh
 Package: fail2ban
 Version: 1.1.0-2
 Architecture: all
 Maintainer: Debian Python Team <email address hidden>
 Installed-Size: 2190
 Depends: python3:any, python3-systemd
 Recommends: nftables | iptables, whois, python3-pyinotify
 Suggests: mailx, system-log-daemon, monit, sqlite3
 Section: net

There is no this runtime dependency on oracular, so either fix first oracular and then backport, or find a way to not runtime depend on it.

[Åka Sikrom (akrosikam)]

OK, I officially give up. Good luck to whomever bothers chasing your constantly-moving target.

[Gianfranco Costamagna (costamagnagianfranco)]

confirmed python3-setuptools not being installed with 1.1.0-2 on noble and oracular...

[Gianfranco Costamagna (costamagnagianfranco)]

@chris, please check oracular or debian sid, and in that case let me know. If setuptools is a runtime dependency, this means that devel release has to be fixed before getting the fix in noble.

[Gianfranco Costamagna (costamagnagianfranco)]

>OK, I officially give up. Good luck to whomever bothers chasing your constantly-moving target.

your patch was good, just the rule is to first fix devel, or mark it as non affected. For an LTS, adding runtime dependencies has to be done carefully, so this is the reason I'm pedantic, to avoid Release Team to reject uploads.

[Gianfranco Costamagna (costamagnagianfranco)]

I went through the code, setuptools is needed due to distutils removals

./fail2ban/server/filtersystemd.py:from distutils.version import LooseVersion
./fail2ban/server/filterpyinotify.py:from distutils.version import LooseVersion

Reuploaded, thanks @akrosikam for the help!

[Gianfranco Costamagna (costamagnagianfranco)]

As said, fixing oracular first.

[Launchpad Janitor (janitor)]

This bug was fixed in the package fail2ban - 1.1.0-2ubuntu2

---------------
fail2ban (1.1.0-2ubuntu2) oracular; urgency=medium

   [ Åka Sikrom (akrosikam) ]
   * Add python3-setuptools runtime dependency needed for Python3.12
     to use distutils.version (LP: #2055114)

 -- Gianfranco Costamagna <email address hidden> Tue, 21 May 2024 09:43:51 +0200

[Chris Halse Rogers (raof)]

An upload of fail2ban to noble-proposed has been rejected from the upload queue for the following reason: "The added source files have a license that needs recording in debian/copyright in order to be distributable".

[Zhang Huangbin (michaelbibby)]

Dear all,

This is a remind that new Fail2ban package (1.1.0-2ubuntu2) is still not available in official apt repo.
Could anyone take care of this?

[Ghadi Rahme (ghadi-rahme)]

Hello all,

I have written a patch for noble backporting two required commits:
- https://github.com/fail2ban/fail2ban/commit/054e1d89ca3f
- https://github.com/fail2ban/fail2ban/commit/77b052fdea51

I also updated the copyright file to reflect the license of the newly added files.
I noticed that the already present copyright file was not following the guidelines defined in the debian docs (https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/) so I tried to follow the format that was already present and added some fields defined in the debian docs.

[Richard Laager (rlaager)]

@ghadi-rahme:

The version in the changelog is wrong. You have "1.0.2-ubuntu1", which should presumably be "1.0.2-3ubuntu1". You are missing the "3" after the dash.

Also, configure-setup-to-install-fail2ban.compat.patch does not apply cleanly. Your version has spaces throughout the whole patch (both the context lines and the line you are adding), where the code in the package uses tabs.

[Ghadi Rahme (ghadi-rahme)]

Hello @rlaager

Sorry for the oversight. I have corrected the version number and fixed the inconsistencies in configure-setup-to-install-fail2ban.compat.patch.

I had copied the output of the debdiff command from the terminal into a text file before uploading the debdiff, this converted the tabs into spaces which explains why even the context lines that were auto generated were also using spaces. This time I transferred the file from my container to my system to avoid a repeat of the issue.

[Jure Sah (dustwolfy)]

python3-setuptools was already installed in my case.

workaround worked for me:

wget https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1/+build/28291332/+files/fail2ban_1.1.0-1_all.deb
sudo dpkg -i fail2ban_1.1.0-1_all.deb

I know this is probably not helpful for getting the bug resolved, but I just thought the conversation was confusing for everyone who got here via Google and just wanted a fix.

[Zhang Huangbin (michaelbibby)]

The error is `ERROR No module named 'asynchat'`. This causes fail2ban failed to start on Ubuntu 24.04.

Module `asynchat` and `asyncore` have been removed since Python 3.12, so all Python applications which require `asynchat` and/or `asyncore` must either ship them internally, or ask users to install them manually.

- https://pypi.org/project/pyasynchat/
- https://pypi.org/project/pyasyncore/

These 2 packages are mentioned in Fail2ban README document:
https://github.com/fail2ban/fail2ban/blob/master/README.md?plain=1#L46

And Fail2ban-1.1.0 ships them internally:
https://github.com/fail2ban/fail2ban/tree/master/fail2ban/compat

Original bug report is all about "asynchat", but it's not yet fixed.

[Gianfranco Costamagna (costamagnagianfranco)]

guys, please stop spamming this bug.
The patch is already in noble unapproved queue, waiting for ubuntu-SRU release managers to pick it up.
https://launchpadlibrarian.net/731722634/fail2ban_1.0.2-3_1.0.2-3ubuntu1.24.04.1.diff.gz

https://launchpad.net/ubuntu/noble/+queue?queue_state=1&queue_text=fail

The versioning looks correct 1.0.2-3ubuntu1.24.04.1, I hope it is just a matter of time.

[Richard Laager (rlaager)]

I tested (rebuilt in a PPA) the version from: https://launchpadlibrarian.net/731722634/fail2ban_1.0.2-3_1.0.2-3ubuntu1.24.04.1.diff.gz

It works for me. I can't mark this verification-done, as I didn't use the actual version from -proposed (since it isn't available there yet).

[Mauricio Faria de Oliveira (mfo)]

The vendorized asyncore.py is identical in python3-pyasyncore (main).

It's better to rely on the package (vs. vendoring) for the stable release,
specially for an LTS, where ESM is 10-12 years, to avoid code duplication
and monitoring/patching on potential security vulnerabilities in 2 places.

Also, in the future, if upstream does migrate from asyncore to asyncio,
the compat module might be abandoned and we'd lose updates/fixes, if any;
and, if this were to happen, again, it's potential bug fixes in 2 places.

I discussed the approach with Steve L., and he confirmed this to be fine.

So, I adjusted the existing 2 uploads, build/start tested in PPA/locally,
and confirmed the asyncore module comes from python3-pyasyncore, and the
asynchat module comes from fail2ban compat/vendoring.

Re-uploading to Noble.

...

$ sudo apt install -y python3-pyasyncore
$ dpkg -S /usr/lib/python3/dist-packages/asyncore/asyncore.py
python3-pyasyncore: /usr/lib/python3/dist-packages/asyncore/asyncore.py
$ wget 'https://raw.githubusercontent.com/fail2ban/fail2ban/054e1d89ca3fa8b767ee21db1a3368f3d890baa8/fail2ban/compat/asyncore.py'
$ md5sum asyncore.py /usr/lib/python3/dist-packages/asyncore/asyncore.py
b4d0825373f6b54927cd50763ccfb641 asyncore.py
b4d0825373f6b54927cd50763ccfb641 /usr/lib/python3/dist-packages/asyncore/asyncore.py

...

PPA build with -proposed enabled:
https://launchpad.net/~mfo/+archive/ubuntu/lp2055114

$ systemctl status --full --no-pager fail2ban.service
...
     Active: active (running) since Mon 2024-06-10 19:33:55 -03; 7s ago
...

Jun 10 19:33:55 noble-fail2ban systemd[1]: Started fail2ban.service - Fail2Ban Service.
Jun 10 19:33:55 noble-fail2ban fail2ban-server[10839]: 2024-06-10 22:33:55,664 fail2ban.configreader [10839]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'
Jun 10 19:33:55 noble-fail2ban fail2ban-server[10839]: Server ready

...

$ sudo systemctl stop fail2ban.service

$ sudo strace -e openat /usr/bin/python3 /usr/bin/fail2ban-server -xf start 2>&1 | grep -F -e asynchat. -e asyncore.
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/asyncore/__pycache__/asyncore.cpython-312.pyc", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/fail2ban/compat/__pycache__/asynchat.cpython-312.pyc", O_RDONLY|O_CLOEXEC) = 3

[Brian Murray (brian-murray)]

Hello Robert, or anyone else affected,

Accepted fail2ban into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fail2ban/1.0.2-3ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Robert Dinse (nanook)]

      The new package has been working good for me EXCEPT wordpress thinks it
isn't running even though it is AND it acts on the WordPress filters and jails
I have installed.

-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-
  Eskimo North Linux Friendly Internet Access, Shell Accounts, and Hosting.
    Knowledgeable human assistance, not telephone trees or script readers.
  See our web site: http://www.eskimo.com/ [phone] or [phone].

On Tue, 11 Jun 2024, Brian Murray wrote:

> Date: Tue, 11 Jun 2024 20:05:13 -0000
> From: Brian Murray <email address hidden>
> To: <email address hidden>
> Subject: [Bug 2055114] Re: fail2ban is broken in 24.04 Noble
>
> Hello Robert, or anyone else affected,
>
> Accepted fail2ban into noble-proposed. The package will build now and be
> available at
> https://launchpad.net/ubuntu/+source/fail2ban/1.0.2-3ubuntu0.1 in a few
> hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested, what testing has been
> performed on the package and change the tag from verification-needed-
> noble to verification-done-noble. If it does not fix the bug for you,
> please add a comment stating that, and change the tag to verification-
> failed-noble. In either case, without details of your testing we will
> not be able to proceed.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> ** Changed in: fail2ban (Ubuntu Noble)
> Status: Confirmed => Fix Committed
>
> ** Tags added: verification-needed verification-needed-noble
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2055114
>
> Title:
> fail2ban is broken in 24.04 Noble
>
> Status in fail2ban package in Ubuntu:
> Fix Released
> Status in fail2ban source package in Noble:
> Fix Committed
> Status in fail2ban source package in Oracular:
> Fix Released
>
> Bug description:
> [ Impact ]
> * fail2ban fails to start on Ubuntu 24.04, due to new Python and deprecated classes (now vendored upstream)
>
> [ Test Plan ]
> * Install and check systemctl status fail2ban.
>
> [ Where problems could occur ]
> * Nowhere, fixing a program failing to start
>
> [ Other info ]
> Was working in 22.04, but after upgrading to 24.04 I get this:
>
> × fail2ban.service - Fail2Ban Service
>      Loaded: loaded (/etc/systemd/system/fail2ban.service; enabled; preset: enabled)
>      Active: failed (Result: exit-code) since Sun 2024-02-25 02:27:29 PST; 1 day 18h ago
>    Duration: 2.330s
>        Docs: man:fail2ban(1)...

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted fail2ban (1.0.2-3ubuntu0.1) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

freedombox/unknown (s390x)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#fail2ban

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Chris Cowley (chris-cowley)]

I've just tried the proposed 1.0.2-3ubuntu0.1 package on a clean Noble VM and it fixed the startup issues, as expected, for me.

[Ghadi Rahme (ghadi-rahme)]

About the autopkgtest test failure on s390x.

Based on the logs the issue is not related to fail2ban and is caused by the VM failing to start, the test does not even reach kernel boot. Looks like a flaky test and restarting the test might fix it, it is not a regression

[Chris Garaffa (chrisgaraffa)]

Re # 65, I've applied the update from noble-proposed on a fresh 24.04 installation (created yesterday on Linode) and fail2ban now starts as expected.

[Mauricio Faria de Oliveira (mfo)]

Flipping the verification tags as requested in comment 65,
based on feedback from comments 70, 68, and 66.

(The other issue mentioned in comment 66 seems unrelated,
and might just be something else now that the package is
able to start services at all, and thus hit other issues.)

Retried the autopkgtest in s390x per comment 69 and logs.

[Josh L (keepittech)]

Re # 65, I've tested and even created a video showing people how to install and test it as well . Thanks, for this fix.

[Joris Le Blansch (j-leblansch)]

@brian-murray the wiki mentioned to add proposed is sadly outdated and for 16.04. More recent documents, show this changed radically in 24.04 so that you can cherry pick updates you want, starting with add-apt-repository -yp proposed. This didn't work for me neither.

I did as follows:

- Open /etc/apt/sources.list.d/ubuntu.sources and add noble-proposed
- apt update
- apt-get install -t noble-proposed fail2ban
- fail2ban now works again
- Open /etc/apt/sources.list.d/ubuntu.sources and remove noble-proposed
- apt update

You are now safe to not accidently update other packages from proposed if that's not wanted (ie. on a production system).

Hope this helps someone. It works for me and I can confirm that fail2ban is working again with this approach.

[Daniel Feliciano Branco (danielblues)]

Thanks Joris, that worked for me to.
Best,

[Grant Slater (firefishy)]

Is there an estimate of when the fixed version of fail2ban might land in nobel?

[Tony Snook (tsnook)]

I understand there are workarounds, but it would be very helpful if this bug was resolved before Ubuntu 23.10 (Mantic Minotaur) reaches End of Life on July 11, 2024. After which, those of us running interim releases, will need to upgrade to the 24.04 Noble release.

[Michael Kofler (michael-kofler)]

Ubuntu LTS promises updates for five years. Ubuntu Pro increases this to 10 or even 12 years.

fail2ban is a pretty critical security component. It failed out of the box in Ubuntu 24.04. More than 2 months have passed and still no fix has been released.

This is NOT a specific criticism of the maintainers of fail2ban -- I am sure they are doing what they can, and the whole situation is somewhat unfortunate. The fix is imminent anyway.

But the bigger question remains: Should I trust Ubuntus/Canonicals update guarantees for a decade when they fail within the first few months? Not on an exotic package, but on a fairly mainstream security component? Is Ubuntu Server ready to go? I have had excellent experiences with Ubuntu Server in the past, but I have my doubts about the future.

[Richard Laager (rlaager)]

Note that fail2ban is in universe, not main. This was surprising to me, and something I only realized because of this bug. I too think of fail2ban as a core security component. I wish Ubuntu would promote it to main, but that's a different conversation.

Traditionally, being in universe has meant that support is "best effort". In my opinion, that was generally security/CVE support at most. Canonical has recently announced expanded support of packages in universe, which is great. However, I share your concern that they may not be able to keep up with all of the packages in universe. Time will tell.

[fredbcode (fredbcode)]

I tried this package, https://launchpad.net/ubuntu/+source/fail2ban (proposed) on a server without fail2ban

I missed something ? I guess this is related to https://github.com/fail2ban/fail2ban/issues/3754
But if some dependencies are missed i should have some warning messages from package, no ?

dpkg -i fail2ban_1.0.2-3ubuntu0.1_all.deb
(Lecture de la base de données... 190469 fichiers et répertoires déjà installés.)
Préparation du dépaquetage de fail2ban_1.0.2-3ubuntu0.1_all.deb ...
Dépaquetage de fail2ban (1.0.2-3ubuntu0.1) sur (1.0.2-3) ...
Paramétrage de fail2ban (1.0.2-3ubuntu0.1) ...
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:224: SyntaxWarning: invalid escape sequence '\s'
  "1490349000 test failed.dns.ch", "^\s*test <F-ID>\S+</F-ID>"
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:435: SyntaxWarning: invalid escape sequence '\S'
  '^'+prefix+'<F-ID>User <F-USER>\S+</F-USER></F-ID> not allowed\n'
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:443: SyntaxWarning: invalid escape sequence '\S'
  '^'+prefix+'User <F-USER>\S+</F-USER> not allowed\n'
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:444: SyntaxWarning: invalid escape sequence '\d'
  '^'+prefix+'Received disconnect from <F-ID><ADDR> port \d+</F-ID>'
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:451: SyntaxWarning: invalid escape sequence '\s'
  _test_variants('common', prefix="\s*\S+ sshd\[<F-MLFID>\d+</F-MLFID>\]:\s+")
/usr/lib/python3/dist-packages/fail2ban/tests/fail2banregextestcase.py:537: SyntaxWarning: invalid escape sequence '\['
  'common[prefregex="^svc\[<F-MLFID>\d+</F-MLFID>\] connect <F-CONTENT>.+</F-CONTENT>$"'
/usr/lib/python3/dist-packages/fail2ban/tests/servertestcase.py:1375: SyntaxWarning: invalid escape sequence '\s'
  "`{ nft -a list chain inet f2b-table f2b-chain | grep -oP '@addr-set-j-w-nft-mp\s+.*\s+\Khandle\s+(\d+)$'; } | while read -r hdl; do`",
/usr/lib/python3/dist-packages/fail2ban/tests/servertestcase.py:1378: SyntaxWarning: invalid escape sequence '\s'
  "`{ nft -a list chain inet f2b-table f2b-chain | grep -oP '@addr6-set-j-w-nft-mp\s+.*\s+\Khandle\s+(\d+)$'; } | while read -r hdl; do`",
/usr/lib/python3/dist-packages/fail2ban/tests/servertestcase.py:1421: SyntaxWarning: invalid escape sequence '\s'
  "`{ nft -a list chain inet f2b-table f2b-chain | grep -oP '@addr-set-j-w-nft-ap\s+.*\s+\Khandle\s+(\d+)$'; } | while read -r hdl; do`",
/usr/lib/python3/dist-packages/fail2ban/tests/servertestcase.py:1424: SyntaxWarning: invalid escape sequence '\s'
  "`{ nft -a list chain inet f2b-table f2b-chain | grep -oP '@addr6-set-j-w-nft-ap\s+.*\s+\Khandle\s+(\d+)$'; } | while read -r hdl; do`",
Traitement des actions différées (« triggers ») pour man-db (2.12.0-4build2) ...

[Joris Le Blansch (j-leblansch)]

See my "workaround" above to get the package from proposed. That works for me and apparently for others also.

[fredbcode (fredbcode)]

Yes Joris, my concern is just there is something wrong with the proposed package, no ? I mean I should at least see some messages about missing dependencies

[Ghadi Rahme (ghadi-rahme)]

These are warning and won't affect the functionality of fail2ban. They also appear in the version prior to proposed and are unrelated to the missing dependencies. Please run "$ systemctl status fail2ban" and confirm if fail2ban has an active status when installed from proposed if you would like to verify if the patch resolved the issue.

From the warning messages I can tell they originate from unit test files and are caused by the python interpreter failing to understand escape sequences for expressions that will be ran by the shell and not by python.

Hope this helps clarify things.

[Gianfranco Costamagna (costamagnagianfranco)]

@michael-kofler, the new LTS was released 2 months ago, and I don't expect people running server to update that soon after a release is made. We need some testing, and the amount of bugfixes that came after release is just too much for SRU team to handle them all. Moreover we require at least 7 days of testing before pushing an SRU to release, and this SRU was difficult due to many upstream changes that needed different checks, and different point of views of what is best for the LTS lifespan.

[Chris Cowley (chris-cowley)]

lol, this is comical. The problem was identified 4 months ago. The precise cause and a working patch submitted shortly afterwards. I'm unsubscribing from the bug now. Reading this comment train should tell anyone all they need to know about the Ubuntu support process. It's been an education.

[Launchpad Janitor (janitor)]

This bug was fixed in the package fail2ban - 1.0.2-3ubuntu0.1

---------------
fail2ban (1.0.2-3ubuntu0.1) noble; urgency=medium

  [ Åka Sikrom, Ghadi Elie Rahme]
  * Fix errors with Python 3.12 (LP: #2055114)
    - d/p/lp2055114-1-bundling-async-modules-removed-in-python312.patch:
      add vendoring of asyncore and asynchat as compat modules
    - d/p/lp2055114-2-setup-install-fail2ban-compat.patch:
      install the compat modules
    - d/copyright: add copyright/license for async modules
    - d/control: add Depends: python3-setuptools for distutils.version

  [ Mauricio Faria de Oliveira ]
  * Drop vendoring of asyncore (LP: #2055114)
    - d/p/lp2055114-1-bundling-async-modules-removed-in-python312.patch:
      backport: remove asyncore file and fallback usage
    - d/copyright: adjust license/changes similarly to existing contents
    - d/control: add Depends: python3-pyasyncore

 -- Mauricio Faria de Oliveira <email address hidden> Mon, 10 Jun 2024 18:27:45 -0300

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for fail2ban has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Joris Le Blansch (j-leblansch)]

@fredbcode no I don't think there is anything wrong. If you install through apt or apt-get you will get notified of eventual missing dependencies. If you install through dpkg -i you are on your own to figure that out. This is nothing new of course. So once again, refer to my method to install it and it will work.

[Speculos (speculos)]

Hi all,

I have been able to update the package on two systems using 24.04 with apt upgrade.
I had to force one of them by apt upgrade fail2ban to force the update, as it has been delayed due to the phasing.

The package is the following

fail2ban/noble-updates 1.0.2-3ubuntu0.1 all [pouvant être mis à jour depuis : 1.0.2-3]

Thanks to all the people involved in the patch.

See ya.