I have a set up in MAAS which defines 4 ehternet devices:
- ens255f0 - single untagged vlan, single space (172.16.7.0/24)
- ens255f1 - two vlans (2910, 2912) (172.16.10.0/24, 172.16.12.0/24)
- bond0 - bond of two devices (ens3f0 and ens3f1) (172.16.11.0/24)
When deployed with MAAS, all interfaces come up cleanly. For instance, one can see that from cloud-init-output.log:
Cloud-init v. 0.7.5 running 'init' at Fri, 10 Mar 2017 14:46:15 +0000. Up 8.55 seconds.
ci-info: ++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++
ci-info: +---------------+------+---------------+---------------+-------------------+
ci-info: | Device | Up | Address | Mask | Hw-Address |
ci-info: +---------------+------+---------------+---------------+-------------------+
ci-info: | bond0 | True | . | . | 2c:60:0c:84:32:a5 |
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . |
ci-info: | ens255f1.2912 | True | 172.16.12.252 | 255.255.255.0 | 2c:60:0c:f9:74:bc |
ci-info: | ens255f1.2910 | True | 172.16.10.252 | 255.255.255.0 | 2c:60:0c:f9:74:bc |
ci-info: | bond0.2911 | True | 172.16.11.252 | 255.255.255.0 | 2c:60:0c:84:32:a5 |
ci-info: | ens3f1 | True | . | . | 2c:60:0c:84:32:a5 |
ci-info: | ens3f0 | True | . | . | 2c:60:0c:84:32:a5 |
ci-info: | ens255f0 | True | 172.16.7.106 | 255.255.255.0 | 2c:60:0c:f9:74:bb |
ci-info: | ens255f1 | True | . | . | 2c:60:0c:f9:74:bc |
ci-info: +---------------+------+---------------+---------------+-------------------+
ci-info: +++++++++++++++++++++++++++++++++Route info+++++++++++++++++++++++++++++++++
ci-info: +-------+-------------+------------+---------------+---------------+-------+
ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
ci-info: +-------+-------------+------------+---------------+---------------+-------+
ci-info: | 0 | 0.0.0.0 | 172.16.7.1 | 0.0.0.0 | ens255f0 | UG |
ci-info: | 1 | 172.16.7.0 | 0.0.0.0 | 255.255.255.0 | ens255f0 | U |
ci-info: | 2 | 172.16.10.0 | 0.0.0.0 | 255.255.255.0 | ens255f1.2910 | U |
ci-info: | 3 | 172.16.11.0 | 0.0.0.0 | 255.255.255.0 | bond0.2911 | U |
ci-info: | 4 | 172.16.12.0 | 0.0.0.0 | 255.255.255.0 | ens255f1.2912 | U |
ci-info: +-------+-------------+------------+---------------+---------------+-------+
When deploying container, which is connected to multiple spaces, connected to these three intercaces, juju rearranges the /etc/network/interfaces and it does look correct, but it fails to bring up the interfaces. Juju log also says:
2017-03-10 14:49:48 ERROR juju.provisioner provisioner_task.go:707 cannot start instance for machine "0/lxd/1": unable to setup network: host machine "0" has no available device in space(s) "ceph-access-space"
2017-03-10 14:54:21 ERROR juju.provisioner provisioner_task.go:707 cannot start instance for machine "0/lxd/0": unable to find host bridge for space(s) "internal-space" for container "0/lxd/0"
What is interesting is that it fails on every machine with the same setup. But it works on one other machine. This machine however, has different NIC setup. ens3f* and ens255f* are switched, so bond is created from ens255f0 and ens255f1, while VLANs are on ens3f1, and untagged vlan is on ens3f0.