improperly mapped source privileged ports may allow for obtaining privileged resources on the host

[Jeremy Nickurak]

ibvirt creates iptables rules when
guest systems are setup for masquerading. The iptables rule will be of the
following format:

# iptables-save -t nat
# Generated by iptables-save v1.4.7 on Wed Jun 9 14:59:03 2010
*nat
:PREROUTING ACCEPT [45:5146]
:POSTROUTING ACCEPT [889:54117]
:OUTPUT ACCEPT [889:54117]
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Wed Jun 9 14:59:03 2010

With masquerading, outgoing connections will have their source-port mapped to a
NAT-selected port, and the iptables default is for privileged ports to be
mapped to privileged (<1024) ports.

This will allow users in guests that have root privileges (in the guest,
independent of their privileges on the host), to obtain privileged resources on
the host, as well as being able to access other resources by using it. One
example is with standard NFS exports now being accessible to the guests,
because the guests will appear to be from the same IP as a possibly trusted
host.

To illustrate:

regular-user@host-machine:~$ ssh guest-machine
Last login: Wed Jun 9 11:33:13 2010 from host-machine
regular-user@guest-machine:~$ sudo mount -t nfs -o udp,rw,soft
remote-nfs-server:/some-export /mnt/tmp

(get UID of resources on the remote NFS server, and create a user in the
guest with that UID>
(su to that new UID)

user-with-same-uid@guest-machine:~$ cp -R /mnt/tmp/secret-data /home/mycopy; rm
-Rf /mnt/tmp/secret-data

A normal user account would not normally be able to do something like this
because they cannot bind to the privileged port, however in this case the NAT
rules permits it.

An example modified set of iptables rules that may solve the problem was
supplied as well:

/sbin/iptables -t nat -A POSTROUTING -p tcp -o $(IFACE) -j MASQUERADE
--to-ports 1024-65535
/sbin/iptables -t nat -A POSTROUTING -p udp -o $(IFACE) -j MASQUERADE
--to-ports 1024-65535
/sbin/iptables -t nat -A POSTROUTING -p icmp -o $(IFACE) -j MASQUERADE

The above uses the --to-ports option that forces iptables' masquerading module
to only map guest reqursts to non-privileged ports.

Confirmed in libvirt 0.6.2, as well as libvirt 0.7.7 (by upstream)

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: libvirt0 0.7.5-5ubuntu27
ProcVersionSignature: Ubuntu 2.6.32-22.35-server 2.6.32.11+drm33.2
Uname: Linux 2.6.32-22-server x86_64
Architecture: amd64
Date: Wed Jun 9 15:16:05 2010
ProcEnviron:
 PATH=(custom, no user)
 LANG=en_CA.UTF-8
 SHELL=/bin/bash
SourcePackage: libvirt

[Vincent (vincent-redhat-bugs)]

Jeremy Nickurak reported an issue with how libvirt creates iptables rules when guest systems are setup for masquerading. The iptables rule will be of the following format:

# iptables-save -t nat
# Generated by iptables-save v1.4.7 on Wed Jun 9 14:59:03 2010
*nat
:PREROUTING ACCEPT [45:5146]
:POSTROUTING ACCEPT [889:54117]
:OUTPUT ACCEPT [889:54117]
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Wed Jun 9 14:59:03 2010

With masquerading, outgoing connections will have their source-port mapped to a NAT-selected port, and the iptables default is for privileged ports to be mapped to privileged (<1024) ports.

This will allow users in guests that have root privileges (in the guest, independent of their privileges on the host), to obtain privileged resources on the host, as well as being able to access other resources by using it. One example is with standard NFS exports now being accessible to the guests, because the guests will appear to be from the same IP as a possibly trusted host.

To illustrate:

regular-user@host-machine:~$ ssh guest-machine
Last login: Wed Jun 9 11:33:13 2010 from host-machine
regular-user@guest-machine:~$ sudo mount -t nfs -o udp,rw,soft
remote-nfs-server:/some-export /mnt/tmp

(get UID of resources on the remote NFS server, and create a user in the
guest with that UID>
(su to that new UID)

user-with-same-uid@guest-machine:~$ cp -R /mnt/tmp/secret-data /home/mycopy; rm -Rf /mnt/tmp/secret-data

A normal user account would not normally be able to do something like this because they cannot bind to the privileged port, however in this case the NAT rules permits it.

An example modified set of iptables rules that may solve the problem was supplied as well:

/sbin/iptables -t nat -A POSTROUTING -p tcp -o $(IFACE) -j MASQUERADE --to-ports 1024-65535
/sbin/iptables -t nat -A POSTROUTING -p udp -o $(IFACE) -j MASQUERADE --to-ports 1024-65535
/sbin/iptables -t nat -A POSTROUTING -p icmp -o $(IFACE) -j MASQUERADE

The above uses the --to-ports option that forces iptables' masquerading module to only map guest reqursts to non-privileged ports.

Acknowledgements:

Red Hat would like to thank Jeremy Nickurak for reporting this issue.

[Vincent (vincent-redhat-bugs)]

Jeremy reported this using Fedora 11 (libvirt 0.6.2) using qemu/kvm. I have reproduced this on Fedora 13 (libvirt 0.7.7).

[Daniel (daniel-redhat-bugs)]

A little more detailed scenario for reproducing, assuming:

 - An NFS server (IP == 10.33.8.116 in this example)
 - A virt host (IP == 10.33.8.111 in this example)

On the NFS server setup an exported filesystem, with the 'secure' flag set and restricted to the IP subnet of the virt hosts

  # cat >> /etc/exports <<EOF
  /var/lib/libvirt/images/export 10.33.8.0/255.255.255.0(rw,no_root_squash,secure)
  EOF
  # exportfs -a -r -v

On the virtualization host, ensure the default virtual network is running

  # virsh net-list
  Name State Autostart
  -----------------------------------------
  default active yes

And that it is configured to do NAT

  # virsh net-dumpxml default | grep forward
  <forward mode='nat'/>

Pick any guest OS and make sure it is connected to the default virtual network.

eg its XML config (virsh dumpxml $GUESTNAME) should have a NIC configured to look like this

    <interface type='network'>
      <mac address='52:54:00:56:44:32'/>
      <source network='default'/>
      <model type='virtio'/>
    </interface>

Boot the guest (virsh start $GUESTNAME)

Now, login to the guest in any manner (ssh, virt-viewer, virsh console,etc).

Attempt to mount the NFS server. Due to the NAT flaw, it will succeed.

With the fixed NAT rules it should should fail in this message:

  # mount 10.33.8.116:/var/lib/libvirt/images/export /mnt -t nfs
  mount.nfs: access denied by server while mounting 10.33.8.116:/var/lib/libvirt/images/export

If the NFS server is then modified to set 'insecure' in /etc/exports, the mount should work again, because this disables source port checking.

[Daniel (daniel-redhat-bugs)]

Created attachment 422978
Add a source port mapping for TCP & UDP protocols when masquerading

This patch applies to upstream GIT repo commit 3cf642a0f90551c4bde23fc0d571d1074717b0de

It should apply without trouble to any 0.8.x series libvirt. Older versions will likely need re-diffing, not least because many file locations have changed in the source tree.

[Josh (josh-redhat-bugs)]

I'm not sure if this is a security flaw. It's probably a bug, but I don't see a trust boundary crossed here (it's fuzzy at best).

First you need some sort of network setup where access is tightly controlled. If anyone can connect any random piece of hardware, port based access restrictions don't work.

I presume that if the virtual machine is behind NAT, we're probably talking about someone at a local machine. It's *possible* they would be connecting remotely, we'll talk about that in a bit though. If there is physical hardware access, this is a non issue, there are many other things they can do at this point that don't need virt.

If you trust someone to have root on a VM, but not on the host, you are in trouble. virt is not a security feature. One compromised virt machine can have drastic results for all the others.

In order for this to be a security flaw, you need the condition of a tightly controlled network that allows untrusted users to have root access on a NAT'd guest, who also have access to port controlled resources.

[Daniel (daniel-redhat-bugs)]

> If you trust someone to have root on a VM, but not on the host, you are in
> trouble. virt is not a security feature. One compromised virt machine can have
> drastic results for all the others.

Trusting guest admins with root on their guests is not unreasonable, if you've configured your host OS networking to protect against bad stuff they can do (IP spoofing, MAC address spoofing, etc, etc). This protection can be applied using the libvirt guest NIC filtering capabilities, or manually setup by the admin using iptables. Of course if you're doing that, you can block access to the NFS servers in question that way. So this NAT source port mapping question becomes moot.

> In order for this to be a security flaw, you need the condition of a tightly
> controlled network that allows untrusted users to have root access on a NAT'd
> guest, who also have access to port controlled resources.

I guess the decision here comes down to whether you consider the libvirt NAT based networking capability to be a security feature, as well as a connectivity feature. It wasn't designed as a security feature, rather as a quick way to give access to guests on a laptop using Wifi NICs where bridging is impossible. Any security benefit is at best a convenient side-effect from the NAT-ing of IP packets. We have separate dedicated capabilities for doing network filtering on guest traffic.

Personally, I consider the flaw to be in NFS for expecting source port restrictions to offer any kind of meaningful security, but that's a different topic :-)

[Jeremy (jeremy-redhat-bugs)]

The flaw does effect actual physical NAT set ups as well... but to me it's more obvious that that set up exists, and that the NAT members are accessing privileges resources via the NAT router. With a VM, you click a few buttons, and the NAT is automatic and invisible, so you don't really know that it's happened.

I would however argue that the default for the SNAT and masquerading modules should be to NOT map to privileged ports, since that's really the root of this issue. If you're really in a situation where you have physical control over all the NAT participants, you can opt to open that up.

That would make the changes here a workaround, but push the actual fix to the kernel.

[Vincent (vincent-redhat-bugs)]

This has also been reported to Ubuntu: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/591943

[Petr (petr-redhat-bugs)]

This issue affects libvirt >= 0.2.0.

[Petr (petr-redhat-bugs)]

Created libvirt tracking bugs for this issue

Affects: fedora-all [bug 613625]

[Fedora (fedora-redhat-bugs)]

libvirt-0.8.2-1.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

libvirt-0.8.2-1.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report.

[errata-xmlrpc (errata-xmlrpc-redhat-bugs)]

This issue has been addressed in following products:

  Red Hat Enterprise Linux 5

Via RHSA-2010:0615 https://rhn.redhat.com/errata/RHSA-2010-0615.html

[Vincent (vincent-redhat-bugs)]

Statement:

(none)

[Jamie Strandboge (jdstrand)]

This was fixed in 0.8.3-1ubuntu1.

[Alex Valavanis (valavanisalex)]

Jaunty reached end-of-life on 23 October 2010, so this bug will not be fixed in that version of Ubuntu. It has been fixed in newer versions.

[Jamie Strandboge (jdstrand)]

This was actually fixed in http://www.ubuntu.com/usn/usn-1008-1.

[Cole (cole-redhat-bugs)]

*** Bug 624536 has been marked as a duplicate of this bug. ***