cannot run maas-import-ephemerals inside lxc container

[Scott Moser]

by default lxc containers do not allow mounting ext4 filesystems.
during 'maas-import-ephemerals', maas uses mount to mount an ext4 filesystem-in-a-file to convert the ephemeral image that it downloads from http://maas.ubuntu.com/images into a -root.tar.gz file that curtin will use for installation.

In the end, try to do this, and you'll see something like:

Tue, 03 Dec 2013 17:17:57 +0000: converting /var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010/disk.img to /var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010/dist-root.tar.gz
mount: Could not find any loop device. Maybe this kernel does not know
       about the loop device? (If so, recompile or `modprobe loop'.)
Tue, 03 Dec 2013 17:17:57 +0000: failed to mount /var/lib/maas/ephemeral/precise/ephemeral/amd64/20131010/disk.img

http://askubuntu.com/questions/376345/allow-loop-mounting-files-inside-lxc-containers
describes how to modify a container to allow mount.

It is possible that lxc could allow mount of ext4 filesytems inside a container by default. I will open a task for lxc for that. However, even if it did, the user-namespace'd lxc container that we should have in 14.04 will not work with that.

Attached is instructions on how to recreate this on 12.04 (using cloud-archive). The same should basically work on trusty (using trusty).

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: maas-cluster-controller 1.4+bzr1693+dfsg-0ubuntu2
ProcVersionSignature: Ubuntu 3.12.0-4.12-generic 3.12.1
Uname: Linux 3.12.0-4-generic x86_64
ApportVersion: 2.12.7-0ubuntu1
Architecture: amd64
Date: Tue Dec 3 17:34:18 2013
Ec2AMI: ami-00000611
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: m1.tiny
Ec2Kernel: aki-00000548
Ec2Ramdisk: ari-00000548
PackageArchitecture: all
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: maas
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.maas.maas.local.celeryconfig.cluster.py:
 # UUID identifying the running cluster controller.
 CLUSTER_UUID = '5ccc848a-7a39-4dd5-af61-ad1fa19d183b'
mtime.conffile..etc.maas.maas.cluster.conf: 2013-12-03T17:22:55.003463
mtime.conffile..etc.maas.maas.local.celeryconfig.cluster.py: 2013-12-03T17:22:55.003463
upstart.isc-dhcp-server.override: manual

[Scott Moser (smoser)]

In the interest of having all the info ccontained in this bug, below is what I had written in the ask-ubuntu response.

You can allow your lxc container to do mounts of ext2, ext3, or ext4 filesystems in one of 2 ways. The simplist is to just add the following to the lxc config (/var/lib/lxc/$NAME/config):

lxc.aa_profile = unconfined
lxc.cgroup.devices.allow = b 7:* rwm
lxc.cgroup.devices.allow = c 10:237 rwm

A much more restrictive solution that still grants the necessary permissions is to do the following:

$ sudo tee /etc/apparmor.d/lxc/lxc-custom-mounts <<EOF
# copied and modified from /etc/apparmor.d/lxc/lxc-default
profile lxc-container-extx-mounts flags=(attach_disconnected,mediate_deleted) {
  #include <abstractions/lxc/container-base>
  mount fstype=ext4 -> /**,
  mount fstype=ext3 -> /**,
  mount fstype=ext2 -> /**,
}
EOF

# reload the lxc-containers profile
$ sudo apparmor_parser --replace /etc/apparmor.d/lxc-containers

$ sudo lxc-create -t ubuntu-cloud -n source-saucy-amd64 -- --release=saucy --arch=amd64

$ name="test1"
$ cfg=/var/lib/lxc/$name/config;
$ sudo lxc-clone -o source-saucy-amd64 -n "$name"

## modify the config to use the profile created above
$ sudo grep "#allow-loop" "$cfg" || sudo tee -a "$cfg" <<EOF
#allow-loop
lxc.aa_profile = lxc-container-extx-mounts
lxc.cgroup.devices.allow = b 7:* rwm
lxc.cgroup.devices.allow = c 10:237 rwm
EOF

[Serge Hallyn (serge-hallyn)]

Allowing ext4 mount by default in lxc would require a kernel ('linux' package) or security team task to vet the ext4 superblock parser etc in the kernel.

We can however ship a (optional, non-default) apparmor policy allowing it to make that easier.

[Serge Hallyn (serge-hallyn)]

I'll add an apparmor profile to trusty to allow the fs mounting.

[Serge Hallyn (serge-hallyn)]

This debdiff adds the apparmor profile to allow mounting ext*, xfs and btrfs filesystems inside containers.

Currently the config files are going through some churn. Once that settles down, we could add a comment to the configuration files to

1. set lxc.aa_profile = lxc-container-default-with-nesting
2. add entries to the devices whitelist for either the device, or for loop devices
  ('lxc.cgroup.devices.allow = b 7:* rwm")

I'd like to add an option to specify apparmor profile at create time, but am not yet sure how that's best done - the apparmor profile names are long, and a typo will be annoying.

[Serge Hallyn (serge-hallyn)]

New debdiff.

[Launchpad Janitor (janitor)]

This bug was fixed in the package lxc - 1.0.0~alpha3-0ubuntu7

---------------
lxc (1.0.0~alpha3-0ubuntu7) trusty; urgency=low

  * Add a lxc-default-with-mounting profile which allows the container to
    mount block filesystems. (LP: #1257389)
 -- Serge Hallyn <email address hidden> Mon, 09 Dec 2013 13:19:31 -0600