| 2016-05-20 12:30:28 |
Mikhail Chernik |
description |
Environment: MOS 9.0 ISO 370, hardware lab
Steps to reproduce:
1. Create environment, add a host with >1 NUMA nodes as compute
2. Configure hugepages and DPDK CPU pinning (min 4096 2M hugepages, at least one CPU pinned for DPDK)
3. Turn on DPDK on one interface and move Private network to this interface, deploy cluster
4. Check OVS process CPU utilization and threads on compute, e.g. with "top -n 1 -bH -p `pgrep ovs-vswitchd`"
Expected result:
There are OVS threads with name pmdXX, which fully utilize 1 CPU core each
Actual result:
No pmd threads, error message in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:21.404Z|00021|dpif_netdev|ERR|Cannot create pmd threads due to out of unpinned cores on numa node
Additional information:
PMD threads are successfully started, if NIC and all cores in pmd-cpu-mask are on same NUMA node
Additionally, format of pmd-cpu-mask causes a warning in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:20.779Z|00018|ovs_numa|WARN|Invalid cpu mask: x
I will attach diagnostic snapshot later |
Environment: MOS 9.0 ISO 370, hardware lab
Steps to reproduce:
1. Create environment, add a host with >1 NUMA nodes as compute
2. Configure hugepages and DPDK CPU pinning (min 4096 2M hugepages, at least one CPU pinned for DPDK)
3. Turn on DPDK on one interface and move Private network to this interface, deploy cluster
4. Check OVS process CPU utilization and threads on compute, e.g. with "top -n 1 -bH -p `pgrep ovs-vswitchd`"
Expected result:
There are OVS threads with name pmdXX, which fully utilize 1 CPU core each
Actual result:
No pmd threads, error message in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:21.404Z|00021|dpif_netdev|ERR|Cannot create pmd threads due to out of unpinned cores on numa node
Additional information:
PMD threads are successfully started, if NIC and all cores in pmd-cpu-mask are on same NUMA node
Additionally, format of pmd-cpu-mask causes a warning in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:20.779Z|00018|ovs_numa|WARN|Invalid cpu mask: x
Diagnostic snapshot: http://mos-scale-share.mirantis.com/fuel-snapshot-2016-05-20_08-51-41.tar.xz |
|
