Fuel for OpenStack

  1. Bug #1584006
  2. Comment #0

Comment 0 for bug 1584006

Revision history for this message
Mikhail Chernik (mchernik) wrote : [dpdk]OpenvSwitch pmd threads fail to start due to incorrect cpu pinning if hoast has more than 1 NUMA node

Environment: MOS 9.0 ISO 370, hardware lab

Steps to reproduce:
1. Create environment, add a host with >1 NUMA nodes as compute
2. Configure hugepages and DPDK CPU pinning (min 4096 2M hugepages, at least one CPU pinned for DPDK)
3. Turn on DPDK on one interface and move Private network to this interface, deploy cluster
4. Check OVS process CPU utilization and threads on compute, e.g. with "top -n 1 -bH -p `pgrep ovs-vswitchd`"

Expected result:
There are OVS threads with name pmdXX, which fully utilize 1 CPU core each

Actual result:
No pmd threads, error message in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:21.404Z|00021|dpif_netdev|ERR|Cannot create pmd threads due to out of unpinned cores on numa node

Additional information:

PMD threads are successfully started, if NIC and all cores in pmd-cpu-mask are on same NUMA node

Additionally, format of pmd-cpu-mask causes a warning in /var/log/openwswitch/ovs-vswitchd.log:
2016-05-20T02:51:20.779Z|00018|ovs_numa|WARN|Invalid cpu mask: x

I will attach diagnostic snapshot later