I was informed that the platform tasks affining job is tied to the openstack-compute-node label.
After removing the openstack related labels on both controllers, the platform tasks (except for k8s infra related) seem to be affined correctly. However, some pods failed to launch due to Insufficient Memory. It turned out that Kubernetes allocatable memory is tied to the openstack labels.
When the node has openstack labels, cpu and memory reserved for platform use are "visible" to Kubernetes and thus allocatable to pods. Below is the comparison of 2 controller nodes, one with openstack labels assigned and one without
I was informed that the platform tasks affining job is tied to the openstack- compute- node label.
After removing the openstack related labels on both controllers, the platform tasks (except for k8s infra related) seem to be affined correctly. However, some pods failed to launch due to Insufficient Memory. It turned out that Kubernetes allocatable memory is tied to the openstack labels.
When the node has openstack labels, cpu and memory reserved for platform use are "visible" to Kubernetes and thus allocatable to pods. Below is the comparison of 2 controller nodes, one with openstack labels assigned and one without
Controller-0 (without openstack labels) kubeconfig= /etc/kubernetes /bootstrap- kubelet. conf --kubeconfig= /etc/kubernetes /kubelet. conf --config= /var/lib/ kubelet/ config. yaml --cgroup- driver= cgroupfs --network- plugin= cni --pod-infra- container- image=k8s. gcr.io/ pause:3. 1 --node-ip=face::3 --cpu-manager- policy= none
============
root 117578 1 4 13:05 ? 00:15:45 /usr/bin/kubelet --bootstrap-
Capacity: com/pci_ sriov_net_ group0_ data0: 64 com/pci_ sriov_net_ group0_ data0: 0
cpu: 36
ephemeral-storage: 10190100Ki
hugepages-1Gi: 60Gi
hugepages-2Mi: 0
intel.
memory: 97528444Ki
pods: 110
Allocatable:
cpu: 36 <-- platform and vswitch cpus have not been deducted
ephemeral-storage: 9391196145
hugepages-1Gi: 60Gi
hugepages-2Mi: 0
intel.
memory: 34511484Ki <--- ~32G (platform mem has not been deducted)
pods: 110
controller-1 (without openstack labels) kubeconfig= /etc/kubernetes /bootstrap- kubelet. conf --kubeconfig= /etc/kubernetes /kubelet. conf --config= /var/lib/ kubelet/ config. yaml --cgroup- driver= cgroupfs --network- plugin= cni --pod-infra- container- image=k8s. gcr.io/ pause:3. 1 --node-ip=face::4 --cpu-manager- policy= static --system- reserved- cgroup= /system. slice --system- reserved= cpu=2,memory= 16500Mi
============
root 117762 1 3 15:24 ? 00:06:10 /usr/bin/kubelet --bootstrap-
Capacity: com/pci_ sriov_net_ group0_ data0: 64 com/pci_ sriov_net_ group0_ data0: 64
cpu: 36
ephemeral-storage: 10190100Ki
hugepages-1Gi: 70Gi
hugepages-2Mi: 0
intel.
memory: 97528444Ki
pods: 110
Allocatable:
cpu: 34 <--- vswitch cpus have not been deducted
ephemeral-storage: 9391196145
hugepages-1Gi: 70Gi
hugepages-2Mi: 0
intel.
memory: 7129724Ki <--- ~6G (platform mem has been deducted)
pods: 110