Activity log for bug #2039316

Date Who What changed Old value New value Message
2023-10-13 20:32:35 Pedro Guimarães bug added bug
2023-10-18 06:49:16 Ian Booth juju: status New Triaged
2023-10-18 06:49:20 Ian Booth juju: importance Undecided Wishlist
2023-10-18 06:49:40 Ian Booth tags constraints k8s
2023-10-23 07:09:35 Pedro Guimarães description I was following the instructions discussed in: https://discourse.charmhub.io/t/pod-priority-and-affinity-in-juju-charms/4091/6 Using juju 2.9.45 + single-node microk8s 1.27/stable for this test. However, affinity / anti-affinity does not give the same AZ capabilities we have in the VM world. AZs means I have a list of groups of no common servers. That means an operator can deploy knowing its workloads will land on different nodes, networks, storage, etc. I believe we can get a very close UX as VMs in k8s by using: 1) using nodeAffinity to select the zones set in constraints [2] 2) setting topology spread [1] With these two features enabled, one could set the following: $ juju deploy postgresql-k8s -n3 --constraints="zones=z1,z2 tags=topology-spread.topologyKey=topology.kubernetes.io/zone,topolocyspread.maxSkew=1" That would result in a deployment where: 1) nodeAffinity only allows to select nodes in z1 or z2 and topologyKey is set in tags 2) to spread pods across nodes, respecting the topologyKey and maxSkew of 1 --------------------------------------------------------------------------- I also noticed that "topologyKey" is not being respected, where command: $ juju deploy postgresql-k8s --channel=14/edge -n3 --constraints="tags=anti-pod.topology-key=topology.kubernetes.io/node" Results in: https://pastebin.ubuntu.com/p/8Ys8T4h8ZD/ Where "topology-key" is not set. In any case, I don't believe that will provide the same UX as described above. We should have both options (pod/anti-pod syntax + zones and topologyKeys) -------------------------------------------------------------------------- [1] https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#topologyspreadconstraints-field [2] https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity I was following the instructions discussed in: https://discourse.charmhub.io/t/pod-priority-and-affinity-in-juju-charms/4091/6 Using juju 2.9.45 + single-node microk8s 1.27/stable for this test. However, affinity / anti-affinity does not give the same AZ capabilities we have in the VM world. AZs means I have a list of groups of no common servers. That means an operator can deploy knowing its workloads will land on different nodes, networks, storage, etc. I believe we can get a very close UX as VMs in k8s by using: 1) using nodeAffinity to select the zones set in constraints [2] 2) setting topology spread [1] With these two features enabled, one could set the following: $ juju deploy postgresql-k8s -n3 --constraints="zones=z1,z2 tags=topology-spread.topologyKey=topology.kubernetes.io/zone,topolocyspread.maxSkew=1" That would result in a deployment where: 1) nodeAffinity only allows to select nodes in z1 or z2 and topologyKey is set in tags 2) to spread pods across nodes, respecting the topologyKey and maxSkew of 1 --------------------------------------------------------------------------- I also noticed that "topologyKey" is not being respected, where command: $ juju deploy postgresql-k8s --channel=14/edge -n3 --constraints="tags=anti-pod.topology-key=topology.kubernetes.io/node" Results in: https://pastebin.ubuntu.com/p/8Ys8T4h8ZD/ Where "topology-key" is not set. In any case, I don't believe that will provide the same UX as described above. We should have both options (pod/anti-pod syntax + zones and topologyKeys) This has been reported as a separated bug: https://bugs.launchpad.net/juju/+bug/2040136 -------------------------------------------------------------------------- [1] https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#topologyspreadconstraints-field [2] https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity
2024-01-22 13:59:18 Pedro Guimarães tags constraints k8s canonical-data-platform-eng constraints k8s