Some platform services have excessive number of worker processes

Bug #1850852 reported by Tee Ngo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Tee Ngo

Bug Description

Brief Description
-----------------
Currently most platform services scale their worker processes based on the number of platform cores in standard configuration. However, this number should not exceed 10. It's observed in a lab with 36 core controller, there are 30 keystone-public worker processes, 18 dashboard worker processes, 8 barbican worker processes.

Now that OpenStack is no longer part of the platform, the number of worker processes should follow the following general rule:
- Frequently used services: number of platform cores/2, min=2, max=10 (e.g. keystone)
- Moderately used services: number of platform cores/5, min=2, max=8 (e.g. sysinv, horizon. vim)
- Rarely used service: number of cores/1, min=1, max=3 (e.g. barbican)

Furthermore, in All-in-one configuration, the number of worker processes is currently limited to 3 max. Since the number of workers that can be added in AIODX is not limited, this restriction must be removed and the above rule must also be applied to this configuration as well.

Severity
--------
Major

Steps to Reproduce
------------------
Bring up a standard system with high core controllers

Expected Behavior
------------------
Number of worker processes of platform services should not exceed 10.

Actual Behavior
----------------
Some services have excessive number of worker processes taking up a considerable amount of platform memory and contributing to large cpu spike during swact.

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
Standard system

Branch/Pull Time/Commit
-----------------------
Latest

Last Pass
---------
N/A

Timestamp/Logs
--------------
N/A

Test Activity
-------------
Evaluation

Tags: stx.config
Revision history for this message
Ghada Khalil (gkhalil) wrote :

stx.3.0 / medium priority - this is a tuning exercise

tags: added: stx.3.0 stx.config
Changed in starlingx:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Tee Ngo (teewrs)
Revision history for this message
Ghada Khalil (gkhalil) wrote :

As per agreement with the community, moving unresolved medium priority bugs (< 100 days OR recently reproduced) from stx.3.0 to stx.4.0

tags: added: stx.4.0
removed: stx.3.0
Revision history for this message
Ghada Khalil (gkhalil) wrote :

Lowering the priority - this is still a would-be-nice to fix, but should not gate stx.4.0 given this is also an issue in stx.3.0 and requires a tuning exercise to address.

tags: removed: stx.4.0
Changed in starlingx:
importance: Medium → Low
Changed in starlingx:
status: Triaged → Won't Fix
Revision history for this message
Ramaswamy Subramanian (rsubrama) wrote :

Addressed as part of StoryBoard: 2010087

Changed in starlingx:
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.