wrong pmd-cpu-mask in case of ovs+dpdk

[Sergii]

Detailed bug description:
 When DPDK is enabled, cpus for pmd-cpu-mask used only from NUMA which that physical interface belong to. So, if our interface placed on NUMA1 for example (As in my case on compute node-1), then we need cpus for pmd-cpu-mask only from that NUMA. All the rest will be used for nothing.

Steps to reproduce:
 1. Create cluster, enable dpdk.
 2. set Nova CPU pinning = 30
    set DPDK CPU pinning = 8
 3. Deploy cluster
 4. Check interface placement

    root@node-1:~# cat /etc/dpdk/interfaces
    pci 0000:81:00.0 igb_uio

    root@node-1:~# cat /sys/bus/pci/devices/0000\:81\:00.0/numa_node
    1
    As we can see it is NUMA1

    root@node-1:~# lscpu | grep NUMA
    NUMA node(s): 2
    NUMA node0 CPU(s): 0-9,20-29
    NUMA node1 CPU(s): 10-19,30-39

 5. Check pmd-cpu-mask

    root@node-1:~# ovs-vsctl get open_vswitch . other_config
    {dpdk-extra="-n 2 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664", dpdk-init="true", dpdk-lcore-mask="0x400", dpdk-socket-mem="1024,1024", pmd-cpu-mask="0x1c0101801"}

    pmd-cpu-mask="0x1c0101801" it is cores 0,20,30,11,31,12,32

So, in our case we just lost cores 0 and 20. We can check it with pmd-stats-show:

    root@node-1:~# ovs-appctl dpif-netdev/pmd-stats-show | grep "thread"
    pmd thread numa_id 1 core_id 11:
    pmd thread numa_id 1 core_id 12:
    pmd thread numa_id 1 core_id 30:
    pmd thread numa_id 1 core_id 31:
    pmd thread numa_id 1 core_id 32:
    main thread:

So we don't have 8 cores as we configured before deploying. We have only 6 (5 for pmd threads and 1 for dpdk-lcore)

Expected results:
 cpus from pmd-cpu-mask belong to correct NUMA

Actual result:
 Wrong distribution of cpus. It costs us a lot. We lost cpus

Reproducibility:
 Fuel 10.x, checked on build 1538

Workaround:
 Manualy set pmd-cpu-mask with command:
    ovs-vsctl set open_vswitch . other_config:pmd-cpu-mask="0x"
 But you need to be careful and check "vcpu_pin_set" in nova.conf and check "isolcpus" in /etc/default/grub. You need to analise cpu destribution and understand which CPUs you can use. Or edit all the parameters as needed.

Description of the environment:
    Compute node with 2 sockets. Total 40xIntel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz with enabled Hyper Threading

Fuel diagnostic snapshot:
    http://mos-scale-share.mirantis.com/sgudz/fuel-snapshot-wrong-pmd-cpu-2017-03-30_12-58-52.tar