[RFE] Managing zones in k8s

[Pedro Guimarães]

I was following the instructions discussed in: https://discourse.charmhub.io/t/pod-priority-and-affinity-in-juju-charms/4091/6

Using juju 2.9.45 + single-node microk8s 1.27/stable for this test.

However, affinity / anti-affinity does not give the same AZ capabilities we have in the VM world.
AZs means I have a list of groups of no common servers. That means an operator can deploy knowing its workloads will land on different nodes, networks, storage, etc.

I believe we can get a very close UX as VMs in k8s by using:
1) using nodeAffinity to select the zones set in constraints [2]
2) setting topology spread [1]

With these two features enabled, one could set the following:
$ juju deploy postgresql-k8s -n3 --constraints="zones=z1,z2 tags=topology-spread.topologyKey=topology.kubernetes.io/zone,topolocyspread.maxSkew=1"

That would result in a deployment where:
1) nodeAffinity only allows to select nodes in z1 or z2 and topologyKey is set in tags
2) to spread pods across nodes, respecting the topologyKey and maxSkew of 1

---------------------------------------------------------------------------

I also noticed that "topologyKey" is not being respected, where command:
$ juju deploy postgresql-k8s --channel=14/edge -n3 --constraints="tags=anti-pod.topology-key=topology.kubernetes.io/node"

Results in: https://pastebin.ubuntu.com/p/8Ys8T4h8ZD/
Where "topology-key" is not set.

In any case, I don't believe that will provide the same UX as described above. We should have both options (pod/anti-pod syntax + zones and topologyKeys)

This has been reported as a separated bug: https://bugs.launchpad.net/juju/+bug/2040136

--------------------------------------------------------------------------
[1] https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#topologyspreadconstraints-field
[2] https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity

[Ian Booth (wallyworld)]

We can look to utilise the now standard node labels

topology.kubernetes.io/zone
topology.kubernetes.io/region

to provide out of the box support for zone constraints and spreading workload units across AZ in the same way as we do for machine clouds. This would be done using affinity under the covers but the UX is generic, ie --constraints="zones=z1,z2" and the spread across AZ is done automatically. Juju doesn't have an option for controlling skew as far as I can recall - if we were to add it it would be modelled to work across all clouds.

We have the k8s specific syntax for bespoke node/pod (anti-)affinity based on node tags, and this syntax is supposed to expose topology key also. I am not sure why that isn't doing what's expected.

[John A Meinel (jameinel)]

Note also that you have 2 different spelling (and definitely one is a typo for):
tags=topology-spread.topologyKey=topology.kubernetes.io/zone,topolocyspread.maxSkew=1

'topology-spread' vs 'topolocyspread'

[John A Meinel (jameinel)]

So I believe one issue is that you are using "anti-pod.topology-key" but if I'm reading the code correctly, it is just:
```
 juju deploy foo --constraints "topology-key=topology.kubernetes.io/zone"
```

[Pedro Guimarães (pguimaraes)]

Hi @jameinel, I am using Juju 3.1/stable with microk8s 1.27/stable. That command does not work:

$ juju deploy postgresql-k8s --constraints "topology-key=topology.kubernetes.io/zone" --debug
09:02:13 INFO juju.cmd supercommand.go:56 running juju [3.1.6 f6a66aa91eec620f5ac04a19d8c06bef03ae6228 gc go1.20.8]
09:02:13 DEBUG juju.cmd supercommand.go:57 args: []string{"/snap/juju/24626/bin/juju", "deploy", "postgresql-k8s", "--constraints", "topology-key=topology.kubernetes.io/zone", "--debug"}
ERROR unknown constraint "topology-key"
09:02:13 DEBUG cmd supercommand.go:528 error stack:
github.com/juju/juju/core/constraints.(*Value).setRaw:524: unknown constraint "topology-key"
github.com/juju/juju/core/constraints.ParseWithAliases:398:
github.com/juju/juju/cmd/juju/application.(*DeployCommand).Run:788:

[Pedro Guimarães (pguimaraes)]

I will move the discussions about anti-pod/pod tag not working to another bug

[Pedro Guimarães (pguimaraes)]

Filed here: https://bugs.launchpad.net/juju/+bug/2040136