nested lxc's within a kvm machine are not accessible

[Adam Stokes]

When running the local provider and creating a few kvm machines we are able to deploy lxc containers within those machines, however, they are unresolvable from outside of the parent machine.

An example status output:

environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.18.0.1
    dns-name: localhost
    instance-id: localhost
    series: trusty
  "1":
    agent-state: started
    agent-version: 1.18.0.1
    dns-name: 10.0.3.99
    instance-id: poe-local-machine-1
    series: trusty
    containers:
      1/lxc/0:
        agent-state: started
        agent-version: 1.18.0.1
        dns-name: 10.0.4.205
        instance-id: juju-machine-1-lxc-0
        series: precise
        hardware: arch=amd64
    hardware: arch=amd64 cpu-cores=1 mem=512M root-disk=8192M
  "2":
    agent-state: started
    agent-version: 1.18.0.1
    dns-name: 10.0.3.149
    instance-id: poe-local-machine-2
    series: trusty
    containers:
      2/lxc/0:
        agent-state: started
        agent-version: 1.18.0.1
        dns-name: 10.0.4.7
        instance-id: juju-machine-2-lxc-0
        series: precise
        hardware: arch=amd64
    hardware: arch=amd64 cpu-cores=1 mem=512M root-disk=8192M
services:
  mysql:
    charm: cs:precise/mysql-38
    exposed: false
    relations:
      cluster:
      - mysql
    units:
      mysql/0:
        agent-state: started
        agent-version: 1.18.0.1
        machine: 1/lxc/0
        public-address: 10.0.4.205
  wordpress:
    charm: cs:precise/wordpress-21
    exposed: false
    relations:
      loadbalancer:
      - wordpress
    units:
      wordpress/0:
        agent-state: started
        agent-version: 1.18.0.1
        machine: 2/lxc/0
        open-ports:
        - 80/tcp
        public-address: 10.0.4.7

Our environments.yaml:

default: local
environments:
  local:
   type: local
   container: kvm

By default the kvm machines are using lxcbr0 as the network bridge. As far as I can tell the nested lxc containers are always using lxcbr0 setup within the kvm machine.

Without proper tunneling or forcing lxc to the the network-bridge defined in the juju environment it will be impossible to use nested lxc's within kvm machines.

If mixing lxc/kvms within the same environment was a supported setup we could work around this issue, otherwise, maybe having the deploy --to lxc:1 inherit the network-bridge from the juju environment.

[Curtis Hovey (sinzui)]

Nested containers is not supported, though desired. As you point out the network addressability is the problem.

[John A Meinel (jameinel)]

This is true, but it isn't planned to be supported for the next release. It might be possible to change the LXC instances onto a br0 that is bridged with eth0 to have them be addressable.

[Adam Stokes (adam-stokes)]

Just to clarify nested containers are not supported in KVM machines? Are nested containers supported in a MaaS environment where bare metal machines handle the nested lxc's properly?

[John A Meinel (jameinel)]

They should work fine in MaaS as the same logic should be used to expose
them that we use for LXC containers.

Note that nesting LXC inside KVM inside MaaS probably won't work.

On Fri, Apr 11, 2014 at 8:09 AM, Adam Stokes <email address hidden>wrote:

> Just to clarify nested containers are not supported in KVM machines? Are
> nested containers supported in a MaaS environment where bare metal
> machines handle the nested lxc's properly?
>
> --
> You received this bug notification because you are subscribed to juju-
> core.
> https://bugs.launchpad.net/bugs/1304530
>
> Title:
> nested lxc's within a kvm machine are not accessible
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1304530/+subscriptions
>