high cpu usage on ovn-controller process
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
networking-ovn |
Fix Released
|
High
|
Russell Bryant |
Bug Description
Description :
1. Create 200 routers with 3 networks on each router, With this load cpu usage of ovn-controller is going very high on the hypervisors. This cpu usage is not getting impacted by the number of VMs.
We tested with 2 VMs and 40 VMs on the hypervisor to see how no. of VMs impacting the cpu of ovn-controller. In both the cases ovn-controller cpu is always showing 100%. But reducing number of networks, routers bringing down the cpu usage of ovn-controller
[root@test ~]# top
top - 11:50:38 up 2 days, 1:46, 2 users, load average: 3.01, 17.81, 29.49
Tasks: 639 total, 2 running, 637 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.6 us, 0.2 sy, 0.0 ni, 95.9 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem : 26386313+total, 25805459+free, 5011364 used, 797176 buff/cache
KiB Swap: 1048572 total, 1048572 free, 0 used. 25836371+avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23095 root 20 0 849368 801748 2428 R 100.0 0.3 1346:55 ovn-controller
49774 qemu 20 0 4735692 1.413g 10156 S 100.0 0.6 259:27.55 qemu-kvm
49776 root 20 0 0 0 0 S 22.3 0.0 35:58.28 vhost-49774
312 root 20 0 0 0 0 S 3.0 0.0 0:07.25 ksoftirqd/38
22015 root 10 -10 2298800 382132 9412 S 1.0 0.1 82:13.25 ovs-vswitchd
115 root 20 0 0 0 0 S 0.3 0.0 0:12.19 rcuos/48
1862 root 20 0 285008 14616 6520 S 0.3 0.0 8:08.79 consul
1 root 20 0 192828 7796 2380 S 0.0 0.0 0:05.70 systemd
[root@test ~]# virsh list
Id Name State
-------
6 instance-00016080 running
[root@test ~]#
[root@test ~]# ovs-vsctl show | grep Port | wc -l
950
[root@test ~]#
After instrumenting the OVN code, this appears to be an issue with how physical_run() is constructed. Thus, the fix is likely in OVN itself, and not in networking-ovn. I'll leave this bug open to track it here for now.