UFW blocks libvirt bridged traffic

Bug #573461 reported by Michael Lustfield
72
This bug affects 14 people
Affects Status Importance Assigned to Milestone
ufw (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Binary package hint: ufw

Basically the issue is....
I use KVM for my virtual machines. These machines grab an IP from the LAN and are used from the outside network.
I run 'ufw enable' on the host. From there is doesn't matter how I configure UFW, the traffic to the VM's is killed.

I'm attaching the log of a fairly lengthy conversation in IRC which may help keep me form having to regurgitate everything. I will if that's what's needed.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: ufw 0.30pre1-0ubuntu2
ProcVersionSignature: Ubuntu 2.6.32-21.32-server 2.6.32.11+drm33.2
Uname: Linux 2.6.32-21-server x86_64
NonfreeKernelModules: ksplice_e4o4fyfg_vmlinux_new ksplice_e4o4fyfg
Architecture: amd64
Date: Sat May 1 23:58:56 2010
InstallationMedia: Ubuntu-Server 10.04 "Lucid Lynx" - Alpha amd64 (20100404)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: ufw

Revision history for this message
Michael Lustfield (michaellustfield) wrote :
summary: - UFW blocks all KVM traffic
+ UFW blocks bridged traffic
Revision history for this message
Jamie Strandboge (jdstrand) wrote : Re: UFW blocks bridged traffic

Seems this is the relevant line in the IRC conversation:
22:24 < MTecknology> cclausen: May 1 22:24:22 pessum kernel: [19981.061455] [UFW BLOCK] IN=br0 OUT=br0 PHYSIN=eth0 PHYSOUT=vnet1 SRC=192.168.3.6 DST=192.168.1.5 LEN=196 TOS=0x10 PREC=0x00 TTL=63 ID=40752 DF PROTO=TCP SPT=55015 DPT=22 WINDOW=126 RES=0x00 ACK PSH URGP=0

Using libvirt with bridging requires additional configuration. For details, see:
http://wiki.libvirt.org/page/Networking#Creating_network_initscripts

Specifically, this section:
"The final step is to disable netfilter on the bridge:

 # cat >> /etc/sysctl.conf <<EOF
 net.bridge.bridge-nf-call-ip6tables = 0
 net.bridge.bridge-nf-call-iptables = 0
 net.bridge.bridge-nf-call-arptables = 0
 EOF
 # sysctl -p /etc/sysctl.conf

It is recommended to do this for performance and security reasons. See Fedora bug #512206. Alternatively you can configure iptables to allow all traffic to be forwarded across the bridge:

# echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged
# lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged
# service libvirtd reload
"

Translated into Ubuntu/ufw language, either:
a) disable netfilter on the bridge via /etc/sysctl.conf, /etc/ufw/sysctl.conf or add a file with the above to /etc/sysctl.d or
b) configure iptables to allow all traffic to be forwarded across the bridge by adding the following to /etc/ufw/before.rules:
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

Changed in ufw (Ubuntu):
status: New → Invalid
summary: - UFW blocks bridged traffic
+ UFW blocks libvirt bridged traffic
Revision history for this message
Michael Lustfield (michaellustfield) wrote :

Just for clarification of what Jamie said. (Breaking it down the the simplest form.)

vim /etc/ufw/before.rules

Add this line
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
after *filter and before COMMIT.

Revision history for this message
Loïc Minier (lool) wrote :

Hmm I had a similar issue but came to a different conclusion.

I had done a libvirt + ufw lucid install and had some vms working -- traffic from vms to the outside world and replies would work, traffic between vms would work etc. Basically things worked fine.

Then I wanted to map a host port to a vm port, that is, I wanted connections from the host on port xyz to go to a VM on port 22 so that I could SSH in to a VM from the Internet despite the libvirt nating on virbr0. I followed the instructions from the ufw-framework man page about Port Redirections, which call for editing the ufw-before-forward rules.

Unfortunately, this wouldn't work and I discovered that the FORWARD rules were something like:
ACCEPT all -- any virbr0 anywhere 192.168.122.0/24 state RELATED,ESTABLISHED
ACCEPT all -- virbr0 any 192.168.122.0/24 anywhere
ACCEPT all -- virbr0 virbr0 anywhere anywhere
REJECT all -- any virbr0 anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- virbr0 any anywhere anywhere reject-with icmp-port-unreachable
ufw-before-logging-forward all -- any any anywhere anywhere
ufw-before-forward all -- any any anywhere anywhere

With these "sanity checks" rules at the very top, to ensure that traffic for 192.168.122.0/24 doesn't come from anywhere else than virbr0. Obviously in my NAT-ing case, this wouldn't be true since the traffic came from eth0 for the host IP but was NATed to the vm's IP.

I moved the ufw-before-logging-forward and ufw-before-forward calls manually before the virbr0 rules, and things worked!

It would seem to me that the issue is with libvirt generated rules which are inserted before the ufw one. This might depend on boot order.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Loic, I think this is a different issue since it appears you are using libvirt's NAT and not libvirt's bridging setup. If I am wrong, please correct me.

Assuming you are using NAT, as you said libvirt is the one doing the inserting of the rules in this case. ufw does have a 'MANAGE_BUILTINS' option in /etc/default/ufw that will flush the builtins, but in this case that wouldn't work too well because libvirt is started after ufw (and so ufw would flush the builtins, setup its rules, then libvirt would come along and insert them at the top again). libvirt probably should grow some functionality to check if ufw is present and enabled, then do something sane. Can you file a wishlist bug against libvirt for this?

Revision history for this message
Loïc Minier (lool) wrote :

Oh right, indeed, that's a NAT setup; I just confused because it's using a bridge as the underlying to do the NAT-ing stuff; I filed bug #595501

Thanks!

Revision history for this message
Luis Mondesi (lemsx1) wrote :

Thanks for the workaround. This should be fixed.

I'm using LXC containers and this bug.

Jan 13 11:25:23 uv09 kernel: [5071210.955631] [UFW BLOCK] IN=br1 OUT=br1 PHYSIN=vethSjyOT7 PHYSOUT=eth1 SRC=172.16.20.10 DST=172.16.20.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=6601 DF PROTO=TCP SPT=41445 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0

Jan 13 11:34:10 uv09 kernel: [5071737.403600] [UFW BLOCK] IN=br1 OUT=br1 PHYSIN=vethSjyOT7 PHYSOUT=eth1 SRC=172.16.20.10 DST=8.8.8.8 LEN=66 TOS=0x00 PREC=0x00 TTL=64 ID=26172 DF PROTO=UDP SPT=60854 DPT=53 LEN=46

Revision history for this message
Derek Simkowiak (ubuntu-cool-st) wrote :

Ubuntu 10.04.1. Both of the above workarounds are not working for me.

I can connect to my server if ufw is disabled with "ufw disable". But as soon as I turn on ufw, I cannot connect to it. I have bridged networking. (I use this server for both KVM and LXC, but during testing I turned off all KVM VMs and LXC containers.)

# On the server:
root@cst6:~# ufw disable
Firewall stopped and disabled on system startup
root@cst6:~# tail -n 3 /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
root@cst6:~# sysctl -p /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
root@cst6:~# tail -n 2 /etc/ufw/before.rules
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
COMMIT
root@cst6:~# ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
root@cst6:~#

# Then I get filtered on my laptop:
derek@derek-laptop:~$ telnet cst6 22
Trying 192.168.2.13...
telnet: Unable to connect to remote host: Connection timed out
derek@derek-laptop:~$

I can see the packets getting filtered in the server /var/log/messages. (192.168.2.10 is the laptop, .2.16 is the server:)

Apr 9 15:53:46 cst6 kernel: [ 1833.009899] [UFW BLOCK] IN=br0 OUT= MAC=94:0c:6d:bc:no:no:no:no:no:no:no:no:no:no SRC=192.168.2.10 DST=192.168.2.13 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=52607 DF PROTO=TCP SPT=41819 DPT=22 WINDOW=5840 RES=0x00 SYN URGP=0

The other UFW is default. I have two bridges, br0 and br1. The one I'm playing with, br0, is DHCP (not that it matters).

Any other solutions? I've looked at my rules with iptables -L but I don't see anything weird.

This is not specific to KVM or libvirt, I don't think. I've taken KVM out of the picture. Any help is appreciated.

Changed in ufw (Ubuntu):
status: Invalid → Confirmed
Revision history for this message
Michael Lustfield (michaellustfield) wrote :

This bug is indeed invalid. I agree that the solution is not immediately apparent. However, read the comments and you will be able to properly configure your system for this setup just as Jaime had to so painfully explain it to me. I'm assuming it was painful for him anyway...

Changed in ufw (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
Nobuto Murata (nobuto) wrote :

Although I don't know this is a right way, setting DEFAULT_FORWARD_POLICY="ACCEPT" in /etc/default/ufw and restarting ufw get through all bridged traffic.

Revision history for this message
Cerin (chrisspen) wrote :

This just started happening to me after a recent apt-get update. It rendered my KVM guests completely inaccessible. The only solution was to disable/uninstall UFW.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

The technique to use to make this work on 12.04 was to edit /etc/ufw/sysctl.conf to have:
# Don't filter packets to our libvirt guests
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

The problem is on later releases of Ubuntu these are not available until the 'bridge' module is loaded. ufw startup happens extremely early in the boot process-- intentionally before networking comes up. If the bridge module is loaded after ufw starts, it set the sysctl values to its default values-- ie, all '1', which breaks libvirt.

What needs to happen on later releases of Ubuntu is to set these values after the bridge module is loaded. There are any number of ways to do this, but since this was related to firewalling, I chose to do it all within ufw:
 * modify /etc/default/ufw to have 'bridge' listed in IPT_MODULES
 * append the following to /etc/ufw/sysctl.conf:
# Don't filter packets to our libvirt guests
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

FYI, I updated the ufw-framework.8 man page to mention libvirt and bridging.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ufw - 0.34~rc-0ubuntu3

---------------
ufw (0.34~rc-0ubuntu3) utopic; urgency=medium

  * debian/patches/0005-lp573461.patch: adjust man page for libvirt
    (LP: #573461)
  * debian/patches/0006-check-for-newer-py3.patch: tests/check-requirements
    updated for newer python3
 -- Jamie Strandboge <email address hidden> Fri, 16 May 2014 15:08:08 -0500

Changed in ufw (Ubuntu):
status: Invalid → Fix Released
Revision history for this message
Stuart Bishop (stub) wrote :

Manually applying the fix in comment #12 gets the juju local provider happy

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.