calico-node and kube-sriov-device-plugin platform pods are requesting CPU resources

Bug #1976300 reported by Mohammad Issa
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Mohammad Issa

Bug Description

Brief Description
-----------------

While running tests in the lab I noticed that the calico-node and kube-sriov-device-plugin pods are requesting CPU resources.  All platform pods are supposed to have no CPU resource requests.

Severity
-----------------

Minor: System/Feature is usable with minor issue

Steps to Reproduce
-----------------

Boot up the current load, run "kubectl describe node <nodename>" for either an AIO node or both controller and worker nodes, then look at the "CPU Requests" column in the table for "Non-terminated Pods".

Expected Behavior
-----------------

There should be no CPU requests for pods in the platform namespaces.

Actual Behavior
-----------------

calico-node pod requested 250m (25% of a CPU) and kube-sriov-device-plugin pod requested 250m (25% of a CPU).

Reproducibility
-----------------

100% Reproducible

System Configuration
-----------------

All

Load info
-----------------

BUILD_ID="2022-05-11_20-00-05"

Last Pass
-----------------

Not sure, it would have passed when we first modified all the platform pods to have no CPU requests.  Not sure what release that was.

Timestamp/Logs
-----------------

N/A

Alarms
-----------------

N/A

Test Activity
-----------------

Developer testing.

Workaround
-----------------

Can still use the system, just can't allocate as many application CPUs as expected.

Mohammad Issa (m-issa)
Changed in starlingx:
assignee: nobody → Mohammad Issa (m-issa)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ansible-playbooks (master)
Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on ansible-playbooks (master)

Change abandoned by "Mohammad Issa <email address hidden>" on branch: master
Review: https://review.opendev.org/c/starlingx/ansible-playbooks/+/844521

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ansible-playbooks (master)

Reviewed: https://review.opendev.org/c/starlingx/ansible-playbooks/+/844329
Committed: https://opendev.org/starlingx/ansible-playbooks/commit/7876bbea76f1210bc95d8a15039a964b638ec261
Submitter: "Zuul (22348)"
Branch: master

commit 7876bbea76f1210bc95d8a15039a964b638ec261
Author: Mohammad Issa <email address hidden>
Date: Wed Jun 1 14:43:35 2022 -0400

    Stop platform pods from requesting CPU resources

    While running tests in the lab, calico-node and
    kube-sriov-device-plugin pods are requesting CPU resources.
    All platform pods are supposed to have no CPU resource requests
    as it messes up the reported CPU allocations and reduces the
    amount of CPU available to the customer applications

    The CPU request allocation for calico-node and
    kube-sriov-device-plugin containers have been changed
    to "0". This will implement no CPU requests for pods
    in the platform namespaces

    The kube-sriov-device-plugin pod CPU requests was modified in
    a SR-IOV enabled lab, to help verify that the code
    change was successful

    Test Plan:

    PASS: set calico-node requests to 0 local build VM
          the following command was executed:
          "kubectl describe node controller-0"
          --> command response:
              Non-terminated Pods:
              .......
              calico-node-kxbpl CPU requests: 0 (0%)

    PASS: set sriovdp-node requests to 0 on SR-IOV enabled lab
          the following command was executed:
          "kubectl describe node controller-0"
          --> command response:
              Non-terminated Pods:
              .......
              kube-sriov-device-plugin-amd64-p5s4g CPU requests: 0 (0%)

    Closes-Bug: 1976300

    Signed-off-by: Mohammad Issa <email address hidden>
    Change-Id: I591886e3dd14cbbef0e0743707eb4cb44dff70a3

Changed in starlingx:
status: In Progress → Fix Released
Mohammad Issa (m-issa)
Changed in starlingx:
status: Fix Released → Opinion
status: Opinion → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.7.0 stx.networking
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.