[2.9] pod/node affinity for sidecar charms not implemented
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Canonical Juju |
Fix Released
|
Wishlist
|
Ian Booth |
Bug Description
Juju version: 2.9.35-ubuntu-amd64
Kubernetes version: v1.24.6
My k8s cluster has some nodes with GPU and some without. I am trying to deploy kubeflow 1.6/stable on only the nodes without a GPU. The nodes without a GPU have a specific label "mldatanode: true". I have tried to follow the examples in the following threads:
- https:/
In my kubeflow bundle I have the following constraints for all the applications:
constraints: tags=mldatanode
I also tried to deploy a single application with the cli:
juju deploy istio-pilot --channel 1.11 --constraints=
Neither of these worked and would put pods on GPU nodes.
- https:/
I also tried to use the hostname of the nodes in the bundle:
to: [kubernetes.
This also did not work as expected and placed pods on GPU nodes.
Furthermore, when creating a model, juju does not support placement directives for the modeloperator pod and I think it should as it ends up on a GPU node.
I will try to work around this by cordoning/
Changed in juju: | |
assignee: | nobody → Thomas Miller (tlmiller) |
Changed in juju: | |
assignee: | Thomas Miller (tlmiller) → Ian Booth (wallyworld) |
Changed in juju: | |
status: | In Progress → Fix Committed |
Changed in juju: | |
status: | Fix Committed → Fix Released |
juju uses the constraint tags that are prefixed with "pod." or "anti-pod." or "node." into pod affinity selectors. "tags=mldatanod e=true, ^mlgpunode= true"
your constraints are: --constraints=
The tag keys don't have the required prefixes to be translated into affinity selection expressions.
A contrived example.
juju deploy somecharm --constraints= "tags=node. foo=a|b| c,^bar= d|e|f,^ foo=g|h, pod.foo= 1|2|3,^ pod.bar= 4|5|6,anti- pod.afoo= x|y|z,^ anti-pod. abar=7| 8|9"
would result in
kubectl get -o json statefulset. apps/somecharm | jq .spec.template. spec.affinity ringSchedulingI gnoredDuringExe cution" : { nodeSelectorTer ms": [
"matchExpres sions": [
"key": "bar",
"operator" : "NotIn",
"values" : [
"d",
"e",
"key": "foo",
"operator" : "NotIn",
"values" : [
"g",
"key": "foo",
"operator" : "In",
"values" : [
"a",
"b", ringSchedulingI gnoredDuringExe cution" : [
"labelSelector ": {
"matchExpres sions": [
"key": "bar",
"operator" : "NotIn",
"values" : [
"4",
"5",
"key": "foo",
"operator" : "In",
"values" : [
"1",
"2",
"topologyKey" : "" ity": { ringSchedulingI gnoredDuringExe cution" : [
"labelSelector ": {
"matchExpres sions": [
"key": "abar",
"operator" : "NotIn",
"values" : [
"7",
"8",
"key": "afoo",
"operator" : "In",
"values" : [
"x",
"y",
"topologyKey" : ""
{
"nodeAffinity": {
"requiredDu
"
{
{
"f"
]
},
{
"h"
]
},
{
"c"
]
}
]
}
]
}
},
"podAffinity": {
"requiredDu
{
{
"6"
]
},
{
"3"
]
}
]
},
}
]
},
"podAntiAffin
"requiredDu
{
{
"9"
]
},
{
"z"
]
}
]
},
}
]
}
}