ipv6 interfaces configured on a machine (in maas) are not added to lxc containers deployed to that machine

[Matt Rae]

I am seeing that lxc containers are not getting configured with interfaces with ipv6 subnets.

The machines hosting containers are configured with 4 interfaces in MAAS. one interface (bond2) has only an ipv6 subnet added and no ipv4. Containers deployed to the host machine are automatically configured bridged to the interfaces on the host machine which have ipv4 subnets. But, there is no interface added to the lxc config bridged to the interface with the ipv6 subnet.

using juju 2.0 beta7 and maas 2.0 beta 6

from the lxc config we see 3 interfaces added, but there should be a 4th interface bridged to br-bond2

# Network configuration
lxc.network.type = veth
lxc.network.link = br-eno1
lxc.network.flags = up
lxc.network.name = eth0
lxc.network.hwaddr = 00:16:3e:b4:54:75
lxc.network.ipv4 = 10.189.69.22/25
lxc.network.ipv4.gateway = 10.189.69.1
lxc.network.mtu = 1500
lxc.network.type = veth
lxc.network.link = br-bond0
lxc.network.flags = up
lxc.network.name = eth1
lxc.network.hwaddr = 00:16:3e:8b:79:c4
lxc.network.ipv4 = 172.27.72.8/26
lxc.network.mtu = 1500
lxc.network.type = veth
lxc.network.link = br-bond1
lxc.network.flags = up
lxc.network.name = eth2
lxc.network.hwaddr = 00:16:3e:6d:4e:b0
lxc.network.ipv4 = 172.27.72.72/26
lxc.network.mtu = 1500

The host has br-bond2 with the ipv6 subnet, but no interface bridged to br-bond2 is getting added to containers deployed on the host

host br-bond2 config:

auto br-bond2
iface br-bond2 inet6 static
address fd0d:ffe0:5771::1:0:0/64
mtu 1500
hwaddress 0c:c4:7a:b7:32:a0
bridge_ports bond2

I'm able to add an interface bridged to br-bond2 to the lxc container manually by editing the container config to add the interface and choosing an available ipv6 address, then restarting the container

lxc.network.type = veth
lxc.network.link = br-bond2
lxc.network.flags = up
lxc.network.name = eth3
lxc.network.ipv6 = fd0d:ffe0:5771::1:0:d/64
lxc.network.mtu = 1500

[Matt Rae (mattrae)]

Attaching screenshot of the host's interface configuration in MAAS

[Andres Rodriguez (andreserl)]

Marking this 'invalid' for MAAS, provided that MAAS only configures the initial /etc/network/interfaces. It is up to Juju to configure the bridges and the interfaces a container gets, and based on that, register the container with IP's in MAAS.

[Dimiter Naydenov (dimitern)]

IPv6 addresses are not yet configured for containers in MAAS, as we needed to gather some feedback on the multi-NIC containers impact with IPv4 first. We'll address this soon.

[James Tunnicliffe (dooferlad)]

Setting to state=confirmed because of Dimiter's comment.
Setting to priority=medium because this is scheduled future work.

[Doug Parrish (dparrish)]

Under Juju 2.1.2, a lxd container was deployed with an IPv4-only interface and a separate IPv6-only interface successfully. However, a dual-stack interface was deployed only with its IPv4 address. The test was conducted with "juju deploy --to lxd:1 --bind 'mgmt ceph=stgfe public=delvrya' glance".

It was also noticed that 'juju add-machine lxd:2' failed to deploy. The LXD host's machine agent log contained the following:

2017-05-03 13:44:55 WARNING juju.provisioner provisioner_task.go:739 failed to start instance (unable to setup network: no obvious space for container "2/lxd/4", host machine has spaces: "delvrya", "delvryb", "mgmt", "stgfe"), retrying in 10s (1 more attempts)
2017-05-03 13:45:05 ERROR juju.provisioner provisioner_task.go:707 cannot start instance for machine "2/lxd/4": unable to setup network: no obvious space for container "2/lxd/4", host machine has spaces: "delvrya", "delvryb", "mgmt", "stgfe"

[Doug Parrish (dparrish)]

Here is the "lxc list", the "ifconfig" and the /e/n/i from both LXD host and container:

Last login: Wed May 3 14:07:46 2017 from 10.1.0.4
ubuntu@novaa:~$ lxc list
+---------------------+---------+--------------------------------+--------------------------------------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------------+---------+--------------------------------+--------------------------------------+------------+-----------+
| juju-e8ba7e-2-lxd-5 | RUNNING | 10.10.0.3 (eth2) | fd4f:0:a:a:216:3eff:fe5e:facc (eth2) | PERSISTENT | 0 |
| | | 10.1.0.207 (eth0) | fd4f:0:a:5:216:3eff:fe45:4d13 (eth1) | | |
+---------------------+---------+--------------------------------+--------------------------------------+------------+-----------+

=== LXD container ===

Note eth1's IPv6 addr differs from what's in its /e/n/i (doesn't help that it's specified as "inet", not "inet6").

root@juju-e8ba7e-2-lxd-5:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3e:75:8a:25
          inet addr:10.1.0.207 Bcast:10.1.0.255 Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe75:8a25/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:28051 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20655 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:173836963 (173.8 MB) TX bytes:2638007 (2.6 MB)

eth1 Link encap:Ethernet HWaddr 00:16:3e:45:4d:13
          inet6 addr: fd4f:0:a:5:216:3eff:fe45:4d13/64 Scope:Global
          inet6 addr: fe80::216:3eff:fe45:4d13/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:3764 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:197946 (197.9 KB) TX bytes:758 (758.0 B)

eth2 Link encap:Ethernet HWaddr 00:16:3e:5e:fa:cc
          inet addr:10.10.0.3 Bcast:10.10.0.255 Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe5e:facc/64 Scope:Link
          inet6 addr: fd4f:0:a:a:216:3eff:fe5e:facc/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:3769 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:198339 (198.3 KB) TX bytes:758 (758.0 B)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:10829 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10829 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:623520 (623.5 KB) TX bytes:623520 (623.5 KB)

lxdbr0 Link encap:Ethernet HWaddr 12:e5:ef:ce:c6:e3
          inet6 addr: fe80::1/64 Scope:Link
          inet6 addr: fe80::10e5:efff:fece:c6...

[Doug Parrish (dparrish)]

The result with 2.2-beta3 is the same as 2.1.2 in comment #6

[Tim Penhey (thumper)]

@Witold, can you please check this? I understand that some other work was done on the container mapping for IP addresses, but I don't know if this was part of the work.

[Doug Parrish (dparrish)]

For Juju 2.2.2, "juju add-machine lxd:8 --constraints spaces=mgmt,stge" stgfe is IPv6-only space.

ubuntu@cmona:/home/ubuntu
$ lxc list
+---------------------+---------+-------------------+--------------------------------------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+---------------------+---------+-------------------+--------------------------------------+------------+-----------+
| juju-e8ba7e-8-lxd-0 | RUNNING | 10.1.0.222 (eth0) | fd4f:0:a:5:216:3eff:feae:c2fa (eth1) | PERSISTENT | 0 |
+---------------------+---------+-------------------+--------------------------------------+------------+-----------+

ubuntu@cmona:/home/ubuntu
$ lxc exec juju-e8ba7e-8-lxd-0 -- bash -o vi
root@juju-e8ba7e-8-lxd-0:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3e:32:4c:5e
          inet addr:10.1.0.222 Bcast:10.1.0.255 Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe32:4c5e/64 Scope:Link
...

eth1 Link encap:Ethernet HWaddr 00:16:3e:ae:c2:fa
          inet6 addr: fe80::216:3eff:feae:c2fa/64 Scope:Link
          inet6 addr: fd4f:0:a:5:216:3eff:feae:c2fa/64 Scope:Global
...

#
# Note "inet", not "inet6", in eth1 stanza
#
root@juju-e8ba7e-8-lxd-0:~# cat /etc/network/interfaces

auto lo eth0 eth1

iface lo inet loopback
  dns-nameservers 10.1.0.4
  dns-search maas virt.ceeness.com

iface eth0 inet static
  address 10.1.0.222/24
  gateway 10.1.0.1

iface eth1 inet static
  address fd4f:0:a:5:0:1:0:3/64

root@juju-e8ba7e-8-lxd-0:~# ll /var/lib/juju/tools
total 12
drwxr-xr-x 3 root root 4096 Aug 1 21:54 ./
drwxr-xr-x 6 root root 4096 Aug 1 21:54 ../
drwxr-xr-x 2 root root 4096 Aug 1 21:54 2.2.2.1-xenial-amd64/
lrwxrwxrwx 1 root root 20 Aug 1 21:54 machine-8-lxd-0 -> 2.2.2.1-xenial-amd64/

[Witold Krecicki (wpk)]

Juju 2.2 has overhauled the whole container networking code, the bug mentioned in the last comment with improper e/n/i generation is fixed https://github.com/juju/juju/pull/7700 here.