Mess in spaces support for Azure

[Radoslaw]

You try to install a bundle on Azure and want machines to use IP addresses from one subnet only. When you have two subnets on Azure standard ways of making machines to use specific subnet fails. One has to use some a bit weird combination.

Tests was made using juju version: 3.1.7-genericlinux-amd64. Configuration after creating a test model:

juju spaces
Name Space ID Subnets
alpha 0 10.253.64.0/24
                 10.253.65.0/26

Assuming we want to machine's to have assigned IP addresses from the first subnet: 10.253.64.0/24

Attempt 1

One can read in docs that alpha the default space. So let's just the second subnet (unanted) to different space:

juju add-space beta 10.253.65.0/26
added space "beta" with subnets 10.253.65.0/26

Now we have such spaces situation:

juju spaces
Name Space ID Subnets
alpha 0 10.253.64.0/24
beta 1 10.253.65.0/26

Effect: It doesn't work. Still beta space is used, machines get addresses randomly from or from another network.

juju machines
Machine State Address Inst id Base AZ Message
0 started 10.253.65.10 machine-0 ubuntu@20.04
1 pending 10.253.64.22 machine-1 ubuntu@20.04

Attempt 2

Move two subnets to two different spaces and point specific space using model's default-space config parameter.

juju add-space gamma 10.253.64.0/24
juju add-space delta 10.253.65.0/26
juju model-config default-space=gamma

So we have such spaces:

juju spaces
Name Space ID Subnets
alpha 0
gamma 1 10.253.64.0/24
delta 2 10.253.65.0/26

It doesn't work, the effect is the same as in Attempt 1 - random addresses from both subnets.

Attempt 3

Configuration of spaces as in Attempt 2 but set constraints to model instead do default-space config:

juju add-space gamma 10.253.64.0/24
juju add-space delta 10.253.65.0/26
juju set-model-constraints -m test allocate-public-ip=false spaces=gamma

It doesn't work. Now you get error saying something about alpha space:

Machine State Address Inst id Base AZ Message
0 down pending ubuntu@20.04 matching subnets to zones: cannot use space "alpha" as deployment target: no subnets

Attempt 4 (finally working)

Combination of attempt 2 and attempt 3, set both: default-space config option and spaces constraint. It finally works. That is a bit strange you have to set both options.

juju add-space gamma 10.253.64.0/24
juju add-space delta 10.253.65.0/26
juju model-config default-space=gamma
juju set-model-constraints -m test allocate-public-ip=false spaces=gamma

All machines get addresses from desired subnet:

Machine State Address Inst id Base AZ Message
0 pending 10.253.64.22 machine-0 ubuntu@20.04
1 pending 10.253.64.23 machine-1 ubuntu@20.04
2 pending 10.253.64.24 machine-2 ubuntu@20.04
3 pending 10.253.64.25 machine-3 ubuntu@20.04
4 pending 10.253.64.26 machine-4 ubuntu@20.04
5 pending 10.253.64.27 machine-5 ubuntu@20.04

[Ian Booth (wallyworld)]

In the scenarios described, can you please include the actual juju commands run to add the machines? The bug says "Effect: It doesn't work" but it's hard to diagnose without knowing how the machines were added or app deployed. ie I am not sure why the default-space setting isn't working without knowing the exact juju commands used to deploy an app or add a machine.

It would be interesting to do another experiment. add-machine and deploy take a constraints argument. If not specified, it falls back to the model constraint.
So, this

juju add-machine --constraints "spaces=gamma"

or this

juju deploy foo --constraints "spaces=gamma"

would have the same effect as adding a model constraint "spaces=gamma" and omitting the constraint with the deploy. I wonder if using the above works for you.

Anyway, if you could provide more info that would be great. I'll mark as Incomplete for now.

[Radoslaw (radraw)]

I added juju commands used for deployment to description as you asked. I used the simplest charm deployment I know for easy of reproducing the issue:

juju deploy ubuntu
juju add-unit ubuntu
...

However I faced the issue originally trying to install charmed-kubernetes bundle.

[Launchpad Janitor (janitor)]

[Expired for Canonical Juju because there has been no activity for 60 days.]