create vm with share cpu_policy breaks exist vm with dedicated cpu_policy when use cpu_allocation

[Jinquan Ni]

1. version
==========

kilo 2015.1.0

3. Reproduce steps:
===================

3.1 environment described
--------------------------

I have one compute node with 32 CPUs. Its NUMA topology is as follows：

    2 sockets * 8 cores * 2 thread = 32

I have configured cpu_allocation_ratio=2 in 'nova.conf'.

3.2 create a flavor with 32 CPUs
--------------------------------

Create a flavor and set NUMA properties as follows:

    $ nova flavor-key 32cpu set hw:numa_nodes=2 \
      hw:numa_cpus.0="0-15" hw:numa_mem.0=256 \
      hw:numa_cpus.1="16-31" hw:numa_mem.1=256 \
      hw:cpu_policy=dedicated

    $ nova flavor-show 32cpu
    +----------------------------+-------+
    | Property | Value |
    +----------------------------+-------+
    | OS-FLV-DISABLED:disabled | False |
    | OS-FLV-EXT-DATA:ephemeral | 0 |
    | disk | 1 |
    | extra_specs | {} |
    | id | 7 |
    | name | 32cpu |
    | os-flavor-access:is_public | True |
    | ram | 512 |
    | rxtx_factor | 1.0 |
    | swap | |
    | vcpus | 32 |
    +----------------------------+-------+

Create a VM using this flavor.

3.3 create another flavor with 20 cpus
--------------------------------------

Create another flavor and set NUMA properties as follows:

    $ nova flavor-key 20cpu set hw:numa_nodes=2 \
      hw:numa_cpus.0="0-9" hw:numa_mem.0=256 \
      hw:numa_cpus.1="10-19" hw:numa_mem.1=256 \
      hw:cpu_policy=share

Create another VM using this flavor.

4 Expected Results
==================

Expected result: create failed
Actual result ： create success

If 'cpu_policy=dedicated' is configured, this should mean the VM has exclusive rights to the specified CPU(s). Even with cpu_allocation, a VM created with the 'shared' CPU policy is sure to use the compute node's CPU resources. As such, the above should not be possible and the VM should fail when exist to schedule.

[Eli Qiao (taget-9)]

I will like to take a look on this, assign it on myself.

[Eli Qiao (taget-9)]

I think you need to make sure how many numa nodes you host has:

I tried in my environment, even I have a cpu like this , but I have only 1 numa node.

Thread(s) per core: 2
Core(s) per socket: 18
Socket(s): 2

ubuntu@jfz1r04h06:~/devstack$ lscpu | grep NUMA
NUMA node(s): 1
NUMA node0 CPU(s): 0-71

So scheduler will fail with this flavor (which you required 2 numa nodes)

nova flavor-key 32cpu set hw:numa_nodes=2 hw:numa_cpus.0="0-15" hw:numa_mem.0=256 hw:numa_cpus.1="16-31" hw:numa_mem.1=256 hw:cpu_policy=dedicated

[Eli Qiao (taget-9)]

BTW, can you tell your libvirt output of virsh capabilities ? I think there maybe a bug from libvirt. check my reported bug here https://bugs.launchpad.net/nova/+bug/1593526

[Jinquan Ni (ni-jinquan)]

Hi, Eli Qiao, yeah, There are two numa nodes on my host:

[root@sbcj-chenling-02-slot11 ~]# lscpu |grep NUMA
NUMA node(s)： 2
NUMA node0 CPU(s)： 0-3,8-11
NUMA node1 CPU(s)： 4-7,12-15

The output of virsh capabilities put in the attach file: virsh-output.txt.
I will have a look the bug : https://bugs.launchpad.net/nova/+bug/1593526.
Thank you!

[Jinquan Ni (ni-jinquan)]

Hi, again, Eli Qiao, i had see the bug you patse, it seems like that is not the same problem.
I leave some comment in the bug link, and i hope that will be useful.
Thank you!

[Eli Qiao (taget-9)]

@Jinquan
thanks for your reply, hmm... libvirt works well, but I see you report this bug Since Liberty(it's EOL).
Have you seen it from latest nova?

[Maciej Szankin (mszankin)]

Removing assignee due to lack of activity.

[Stephen Finucane (stephenfinucane)]

This is not a bug. By definition, instances with the 'shared' have no awareness of the requirements of those with 'dedicated'. You should use host aggregates to separate instances using 'dedicated' policies from the other. Refer to the documentation for more info (the caution)

    http://docs.openstack.org/admin-guide/compute-cpu-topologies.html#customizing-instance-cpu-pinning-policies