Note: Security impact of Libvirt/LXC usage

[Robert Clark]

 if you select libvirt/LXC as your
"virtualization driver", containers are actually not as isolated as you
may think: for example you can affect your own (and others') resource
quotas. See: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1088295

[Robert Clark (robert-clark)]

DRAFT::

Title
-----
Selecting LXC as Nova Virtualiztion Driver can lead to data compromise.

### Summary ###
LXC does not provide the same level of separatoin as hypervisors when chosen as the Nova 'virtualization driver'. Attempting to use LXC as a drop in replacement for a hypervisor can result in data exposure between tenants.

### Affected Services / Software ###
Nova, LXC, Libvirt, 'Virtualization Driver'

### Discussion ###
LXC (also known as Linux containers) is a virtualization technology that works at the operating system level. This is different from hardware virtualization, the approach used by other hypervisors such as KVM, Xen, and VMWare.

The quality of container isolation in LXC heavily depends on implementation. While pure LXC is generally well-isolated through various mechanisms (for example AppArmor in Ubuntu), LXC through libvirt is not.

A guest who operates within one container is able to effect another containers cpu share, memory limit and block devices among other issues.

For more information on the affects of this issue see this [bug] (https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1088295)

### Recommeneded Actions ###
The root of this issue stems from the fact that AppArmour profiles are necessary to enforce container isolation. Such rules exist for normal LXC uses but are not yet enabled for libvirt-lxc.

Currently the OSSG advises that anyone deploying Nova in environments that require any level of separation use a hypervisor such as Xen, KVM, VMware or Hyper-V.

### Contacts / References ###
Nova :
LXC : http://lxc.sourceforge.net/
Libvirt :
Nova Configuration Options :
KVM :
Xen:

[Bryan D. Payne (bdpayne)]

Suggestions:

1) Spell check (e.g., "separatoin" --> "separation")

2) I'd go a little stronger to say that lxc's really aren't designed for security centric tasks... even with an apparmor profile in place. See some discussion at https://wiki.ubuntu.com/LxcSecurity and https://www.berrange.com/posts/2011/09/27/getting-started-with-lxc-using-libvirt/ for additional details.

[Robert Clark (robert-clark)]

1) Wrote this on a plane with my eyes closed, spelling check will follow ;)
2) See next draft

[Robert Clark (robert-clark)]

DRAFT::
Title
-----
Selecting LXC as Nova Virtualization Driver can lead to data compromise.

### Summary ###
LXC does not provide the same level of separation as hypervisors when chosen as the Nova 'virtualization driver'. Attempting to use LXC as a drop in replacement for a hypervisor can result in data exposure between tenants.

### Affected Services / Software ###
Nova, LXC, Libvirt, 'Virtualization Driver'

### Discussion ###
LXC (also known as Linux containers) is a virtualization technology that works at the operating system level. This is different from hardware virtualization, the approach used by other hypervisors such as KVM, Xen, and VMWare.
The quality of container isolation in LXC heavily depends on implementation. While pure LXC is generally well-isolated through various mechanisms (for example AppArmor in Ubuntu), LXC through libvirt is not.
A guest who operates within one container is able to effect another containers cpu share, memory limit and block devices among other issues.
For more information on the affects of this issue see this [bug] (https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1088295)

### Recommended Actions ###
The OSSG advises that anyone deploying Nova in environments that require any level of separation use a hypervisor such as Xen, KVM, VMware or Hyper-V.
LXC security pivots on a system known as DAC (discretionary access control) which is not currently capable of providing strong isolation of guests. Work is underway to improve DAC but it’s not ready for production use at this time.
LXC is not appropriate for enforcing secure separation of guests. Even with appropriate AppArmour policies applied

### Contacts / References ###
Nova : http://docs.openstack.org/developer/nova/
LXC : http://lxc.sourceforge.net/
Libvirt : http://libvirt.org/
KVM : http://www.linux-kvm.org/page/Main_Page
Xen: http://xen.org/products/xenhyp.html
LXC DAC : https://wiki.ubuntu.com/UserNamespace
LXC LibVirt Discussion : https://www.berrange.com/posts/2011/09/27/getting-started-with-lxc-using-libvirt/

[Bryan D. Payne (bdpayne)]

this looks better to me

[Robert Clark (robert-clark)]

Any changes before we publish?

[Bryan D. Payne (bdpayne)]

I'm ok with this, I'll ask for final comments during the meeting this morning.

[Robert Clark (robert-clark)]

Selecting LXC as Nova Virtualization Driver can lead to data compromise.
---------------------------------------------------------------------------------

### Summary ###
LXC does not provide the same level of separation as hypervisors when chosen as the Nova 'virtualization driver'. Attempting to use LXC as a drop in replacement for a hypervisor can result in data exposure between tenants.

### Affected Services / Software ###
Nova, LXC, Libvirt, 'Virtualization Driver'

### Discussion ###
LXC (also known as Linux containers) is a virtualization technology that works at the operating system level. This is different from hardware virtualization, the approach used by other hypervisors such as KVM, Xen, and VMWare.
The quality of container isolation in LXC heavily depends on implementation. While pure LXC is generally well-isolated through various mechanisms (for example AppArmor in Ubuntu), LXC through libvirt is not. A guest who operates within one container is able to effect another containers cpu share, memory limit and block devices among other issues.
For more information on the affects of this issue see this [bug] (https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1088295)

### Recommended Actions ###
The OSSG advises that anyone deploying Nova in environments that require any level of separation use a hypervisor such as Xen, KVM, VMware or Hyper-V.

LXC security pivots on a system known as DAC (discretionary access control) which is not currently capable of providing strong isolation of guests. Work is underway to improve DAC but it’s not ready for production use at this time.

The OSSG recommends against using LXC for enforcing secure separation of guests. Even with appropriate AppArmour policies applied.

### Contacts / References ###
Nova : http://docs.openstack.org/developer/nova/
LXC : http://lxc.sourceforge.net/
Libvirt : http://libvirt.org/
KVM : http://www.linux-kvm.org/page/Main_Page
Xen: http://xen.org/products/xenhyp.html
LXC DAC : https://wiki.ubuntu.com/UserNamespace
LXC LibVirt Discussion : https://www.berrange.com/posts/2011/09/27/getting-started-with-lxc-using-libvirt/

[Robert Clark (robert-clark)]

Suggested change to discussion element:
The Libvirt LXC functionality exposed by OpenStack is built on the kernel
  namespace & cgroup technologies. Until Linux 3.8, there has been no support
  for separate user namespaces in the kernel. As such, there has been no way
  to securely isolate containers from each other or the host environment using
  DAC (discretionary access control). For example, they can escape their resource
  constraints by modifying cgroups settings, or attack the host via various files
  in the proc and sysfs filesystems. The use of MAC (mandatory access control)
  technologies like SELinux or AppArmour can mitigate these problems, but it is
  not practical to write MAC policies that would allow running full OS installs
  in LXC under OpenStack.

  Although initial user namespace support was merged in Linux 3.8, it is not
  yet complete, or mature enough to be considered secure. Work is ongoing to
  finish the kernel namespace support and enhance libvirt LXC to take advantage
  of it.

[Robert Clark (robert-clark)]

The Libvirt LXC functionality exposed by OpenStack is built on the kernel namespace & cgroup technologies. Until Linux 3.8, there has been no support for separate user namespaces in the kernel. As such, there has been no way to securely isolate containers from each other or the host environment using DAC (discretionary access control). For example, they can escape their resource constraints by modifying cgroups settings, or attack the host via various files in the proc and sysfs filesystems. The use of MAC (mandatory access control) technologies like SELinux or AppArmour can mitigate these problems, but it is not practical to write MAC policies that would allow running full OS installs in LXC under OpenStack.

Although initial user namespace support was merged in Linux 3.8, it is not yet complete, or mature enough to be considered secure. Work is ongoing to finish the kernel namespace support and enhance libvirt LXC to take advantage of it.

[Robert Clark (robert-clark)]

Published on OpenStack and OpenStack-Dev