ufw

ufw doesn't support concurrent updates

Bug #1204579 reported by Jeremy on 2013-07-24
26
This bug affects 3 people
Affects Status Importance Assigned to Milestone
ufw
Medium
Jamie Strandboge
ufw (Debian)
Fix Released
Unknown
ufw (Ubuntu)
Medium
Jamie Strandboge
Bionic
Medium
Jamie Strandboge
Cosmic
Medium
Jamie Strandboge
Disco
Medium
Jamie Strandboge

Bug Description

[Impact]

ufw prior to 0.36 did not support locking or use xtables locking which can lead to problems when using ufw as a dynamic firewall.

[Test Case]

$ sudo apt-get install python-minimal # for the test script
$ sudo ./test_ufw_threads.py # this runs for many seconds

A patched ufw will complete with no errors. With an unpatched ufw, there will be many errors like:
$ sudo ./test_ufw_threads.py
...
ERROR: initcaps
[Errno 2] iptables: Chain already exists.

ERROR: initcaps
[Errno 2] iptables: Chain already exists.

[Regression Potential]

ufw 0.36 implements a global exclusive lock before doing any kernel or file writes. Risk of regression includes the lock file not being released due to coding error. The locking is done via fcntl.lockf and ufw releases it in a finally clause that is careful to always release it.

[Other Info]

This locking code has been in production in the ufw snap for years and is known to work is static environments as well as in dynamic environments (IPS/fail2ban/etc).

= Original description =

On a server under Ubuntu 12.04 I automatically update firewall rules and I unfortunately noticed that if 2 rules are processed at the same time it leads to an inconsistent result between iptables and ufw status.
In fact it can be reproduced using the python script in attachment.

Before executing this script I didn't have any rules matching the ip 192.168.254.1 on my computer.
After a launch iptables takes into acount 464 rules.

sudo iptables -L -n | grep 192.168.254.1 | wc -l
464

It should be 500 rules but the script stressed a lot my system and all rules can't be processed by iptables
sudo ./test_ufw_threads.py
iptables: Resource temporarily unavailable.

iptables: Resource temporarily unavailable.

iptables: Resource temporarily unavailable.
[...]

Anyway, ufw stores only the last updates (in /lib/ufw/user.rules):

sudo ufw status
État : actif

Vers Action Depuis
---- ------ ------
192.168.254.1 1234/tcp ALLOW 192.168.46.9
192.168.254.1 1234/tcp ALLOW 192.168.22.10
192.168.254.1 1234/tcp ALLOW 192.168.32.10
192.168.254.1 1234/tcp ALLOW 192.168.5.10
192.168.254.1 1234/tcp ALLOW 192.168.43.9
192.168.254.1 1234/tcp ALLOW 192.168.40.10
192.168.254.1 1234/tcp ALLOW 192.168.46.10
192.168.254.1 1234/tcp ALLOW 192.168.48.10
192.168.254.1 1234/tcp ALLOW 192.168.42.10
192.168.254.1 1234/tcp ALLOW 192.168.43.10

So I can't delete other rules using ufw, I had to directly use iptables.
In this case I can only delete 10 rules using ufw.
Could you please handle some kind of lock?

Jeremy (jeremy-jr) wrote :
description: updated
Changed in ufw (Ubuntu):
status: New → Triaged
Changed in ufw:
status: New → Triaged
Changed in ufw (Ubuntu):
importance: Undecided → Medium
Changed in ufw:
importance: Undecided → Medium
Changed in ufw:
status: Triaged → Fix Committed
milestone: none → 0.34
Changed in ufw:
status: Fix Committed → In Progress
Jamie Strandboge (jdstrand) wrote :

There was a problem with the patch and this will be in the next release of ufw.

Changed in ufw:
milestone: 0.34 → none
Changed in ufw (Debian):
status: Unknown → New

Is there any update on this? I know there is a -w command in iptables with this patch https://patchwork.ozlabs.org/patch/635676/. Is there a way to call through ufw to set this option to increase the waits?

Changed in ufw:
status: In Progress → Triaged
Jamie Strandboge (jdstrand) wrote :

@Christopher - I implemented this a bit differently but had some troubles with the testsuite. I think '-w' would be much simpler. There isn't a way to pass this into ufw, but you could modify backend_iptables.py towards the end of set_rule() to pass the flag.

OK great. I will work on this patch.

Luke (lukepolo) wrote :

Is there a current work around for this, otherwise I have to re-write a bunch of code to go to iptables

Jamie Strandboge (jdstrand) wrote :

@Luke, you could modify backend_iptables.py as mentioned in comment #5 or you could perhaps use 'man 1 flock' or implement a lock wrapper around your invocation of ufw.

That said, since didn't hear back from Christopher, I'll take a look at implementing this for 0.36.

Changed in ufw:
milestone: none → 0.36

Hi all. I posted a question on this last December, but I never received feedback on it. Here is the bug report https://bugs.launchpad.net/ufw/+bug/1652163.

Luke (lukepolo) wrote :

the flock worked perfectly thank you!

Jamie Strandboge (jdstrand) wrote :

This is in git master now and in the ufw snap in candidate.

Changed in ufw:
status: Triaged → Fix Committed
Changed in ufw (Ubuntu):
status: Triaged → In Progress
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in ufw (Ubuntu Cosmic):
status: New → Triaged
Changed in ufw (Ubuntu Bionic):
status: New → Triaged
Changed in ufw (Ubuntu Cosmic):
importance: Undecided → Medium
Changed in ufw (Ubuntu Bionic):
importance: Undecided → Medium
Changed in ufw (Ubuntu Cosmic):
assignee: nobody → Jamie Strandboge (jdstrand)
Changed in ufw (Ubuntu Bionic):
assignee: nobody → Jamie Strandboge (jdstrand)
Jamie Strandboge (jdstrand) wrote :

This is fixed in the new 0.36 release.

Changed in ufw:
assignee: nobody → Jamie Strandboge (jdstrand)
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ufw - 0.36-1

---------------
ufw (0.36-1) unstable; urgency=medium

  * New upstream release (LP: #1782384, LP: #1664133, LP: #1509725,
    LP: #1695718, LP: #1719211, LP: #1775043, LP: #1204579, LP: #1652163,
    LP: #1377600, Closes: 686248, LP: #1368411, LP: #1586258, Closes: 909163,
    Closes: 884932, LP: #1558068)
    - drop 0002-bug849628.patch (included upstream)
    - drop 0003-use-default-tcp-syncookies.patch (included upstream)
    - drop 0004-lp1633698.patch (included upstream)
  * Remaining changes:
    - 0001-optimize-boot.patch
  * debian/ufw.maintscript: remove /etc/bash_completion.d/ufw on upgrade
    (LP: #1602834)
  * debian/control: remove no longer needed xs-python-version and
    x-python3-version fields
  * update debian/before6.rules.md5sum for file shipped in 0.35-6. While both
    before.rules and before6.rules were updated in this new upstream release,
    0.35-6 mistakenly already had its own md5sum for before.rules, so we don't
    need to add it now.

 -- Jamie Strandboge <email address hidden> Fri, 14 Dec 2018 17:50:47 +0000

Changed in ufw (Ubuntu Disco):
status: In Progress → Fix Released
Changed in ufw (Debian):
status: New → Fix Released
description: updated
Changed in ufw (Ubuntu Bionic):
status: Triaged → In Progress
Changed in ufw (Ubuntu Cosmic):
status: Triaged → In Progress

An upload of ufw to cosmic-proposed has been rejected from the upload queue for the following reason: "All bugs mentioned in the .changes file (so therefore also in the new debian/changelog entries) need to comply with SRU standards (test-case, regression potential). Please re-upload after filling out the required info or modify changelog to exclude irrelevant bug numbers.".

Hello Jeremy, or anyone else affected,

Accepted ufw into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ufw/0.36-0ubuntu0.18.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ufw (Ubuntu Cosmic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-cosmic
Changed in ufw (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Brian Murray (brian-murray) wrote :

Hello Jeremy, or anyone else affected,

Accepted ufw into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ufw/0.36-0ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Jamie Strandboge (jdstrand) wrote :

Tested this is fixed in cosmic:

$ apt-cache policy ufw
ufw:
  Installed: 0.36-0ubuntu0.18.10.1
  Candidate: 0.36-0ubuntu0.18.10.1
  Version table:
 *** 0.36-0ubuntu0.18.10.1 500
        500 http://us.archive.ubuntu.com/ubuntu cosmic-proposed/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu cosmic-proposed/main i386 Packages
        100 /var/lib/dpkg/status
     0.35-6 500
        500 http://us.archive.ubuntu.com/ubuntu cosmic/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu cosmic/main i386 Packages

tags: added: verification-done-cosmic
removed: verification-needed-cosmic
tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Jamie Strandboge (jdstrand) wrote :

Verified this is fixed in bionic:

$ apt-cache policy ufw
ufw:
  Installed: 0.36-0ubuntu0.18.04.1
  Candidate: 0.36-0ubuntu0.18.04.1
  Version table:
 *** 0.36-0ubuntu0.18.04.1 500
        500 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu bionic-proposed/main i386 Packages
        100 /var/lib/dpkg/status
     0.35-5 500
        500 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu bionic/main i386 Packages

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ufw - 0.36-0ubuntu0.18.10.1

---------------
ufw (0.36-0ubuntu0.18.10.1) cosmic-proposed; urgency=medium

  * Backport to cosmic addressing the following SRU bugs:
    - LP: #1811129 - master SRU bug
    - LP: #1664133 - before6.rules: echo-reply needs to be before INVALID
    - LP: #1719211 - improve interface name checks
    - LP: #1775043 - shell-completion/bash: adjust for modern bash
    - LP: #1204579 - support concurrent updates
    - LP: #1368411 - add 'prepend' command

 -- Jamie Strandboge <email address hidden> Mon, 25 Mar 2019 20:46:10 +0000

Changed in ufw (Ubuntu Cosmic):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for ufw has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ufw - 0.36-0ubuntu0.18.04.1

---------------
ufw (0.36-0ubuntu0.18.04.1) bionic-proposed; urgency=medium

  * Backport to bionic addressing the following SRU bugs:
    - LP: #1811129 - master SRU bug
    - LP: #1664133 - before6.rules: echo-reply needs to be before INVALID
    - LP: #1719211 - improve interface name checks
    - LP: #1775043 - shell-completion/bash: adjust for modern bash
    - LP: #1204579 - support concurrent updates
    - LP: #1368411 - add 'prepend' command

 -- Jamie Strandboge <email address hidden> Mon, 25 Mar 2019 21:14:25 +0000

Changed in ufw (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.