In AWS using spaces and fan network for a private network does not allow LXC containers to start

Bug #1942950 reported by Xav Paice
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Joseph Phillips

Bug Description

Using Juju 2.9.12

If I use public networks with the subnet allocating public IP addresses, there's no problem. As soon as I start making network spaces in AWS, fan networking doesn't appear to work the way I'd expect.

When I add a private network:
- new subnet, no public IP address allocation
- add a NAT gateway to the public subnet
- add a route table for the private network that points 0.0.0.0/0 at the nat gateway

My new subnet is 10.0.132.0/24.

Added space:

juju add-space oam 10.0.132.0/24
juju reload-spaces

$ juju spaces
Name Space ID Subnets
alpha 0 10.0.129.0/24
                 10.0.130.0/24
                 10.0.131.0/24
                 252.2.0.0/15
                 252.4.0.0/15
                 252.6.0.0/15
oam 1 10.0.132.0/24
                 252.8.0.0/15

Add a new application

juju deploy cs:ubuntu --series focal --constraints "spaces=oam zones=eu-west-2a instance-type=t3.medium" --bind "oam" ubuntu

Machine 12 builds fine, and I can 'juju ssh' to it.

juju deploy cs:ubuntu --to lxd:12 --series focal --constraints "spaces=oam zones=eu-west-2a" --bind "oam" ubuntu-container

Unit fails to build:
machine-12: 23:12:57 INFO juju.container-setup initial container setup with ids: [12/lxd/0]
  machine-12: 23:12:57 INFO juju.packaging.manager Running: snap info lxd
  machine-12: 23:12:57 INFO juju.container.lxd LXD snap is already installed (channel: latest/stable); skipping package installation
  machine-12: 23:12:57 WARNING juju.container-setup not stopping machine agent container watcher due to error: setting up container dependencies on host machine: Not Found
  machine-12: 23:12:57 ERROR juju.container-setup starting container provisioner for lxd: setting up container dependencies on host machine: Not Found

Tried the exact same thing but with --series bionic, and got a different error:

```
machine-14: 00:43:27 WARNING juju.worker.provisioner machine 14/lxd/0 failed to start: unable to setup network: host machine "14" has no available FAN devices in space(s) "oam"

$ juju ssh 14 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 06:58:cc:b6:6d:6c brd ff:ff:ff:ff:ff:ff
    inet 10.0.132.193/24 brd 10.0.132.255 scope global dynamic ens5
       valid_lft 2176sec preferred_lft 2176sec
    inet6 fe80::458:ccff:feb6:6d6c/64 scope link
       valid_lft forever preferred_lft forever
3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 32:6f:32:44:48:fd brd ff:ff:ff:ff:ff:ff
    inet 252.9.130.1/8 scope global fan-252
       valid_lft forever preferred_lft forever
    inet6 fe80::306f:32ff:fe44:48fd/64 scope link
       valid_lft forever preferred_lft forever
4: ftun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8951 qdisc noqueue master fan-252 state UNKNOWN group default qlen 1000
    link/ether 32:6f:32:44:48:fd brd ff:ff:ff:ff:ff:ff
    inet6 fe80::306f:32ff:fe44:48fd/64 scope link
       valid_lft forever preferred_lft forever
5: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 82:a2:04:fd:70:a6 brd ff:ff:ff:ff:ff:ff
    inet 10.233.185.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::80a2:4ff:fefd:70a6/64 scope link
       valid_lft forever preferred_lft forever
Connection to 10.0.132.193 closed.
```

It looks like the fan subnet 252.8.0.0/15 doesn't quite match up with what's configured on the machine 252.9.130.1/8, it's in the range but the subnet masks don't match.

Revision history for this message
Joseph Phillips (manadart) wrote :

The first issue has been raised already here:
https://bugs.launchpad.net/bugs/1942864

For the second, if you run "juju-subnets", does it show that the underlay 10.0.132.0/24 is in the constrained zone eu-west-2a?

Changed in juju:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Joseph Phillips (manadart)
milestone: none → 2.9.14
Changed in juju:
status: Triaged → In Progress
Revision history for this message
Joseph Phillips (manadart) wrote :
Ian Booth (wallyworld)
Changed in juju:
status: In Progress → Fix Committed
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.