Comment 0 for bug 1628504

Description
===========

Our criteria for how we deal with conflicting image metadata and flavor extra specs on both 'cpu_policy' and 'cpu_thread_policy' is rather inconsistent. We should probably configure this such that image metadata is the defacto spec, and an competing flavor metadata causes issues. Original comment below from https://review.openstack.org/#/c/361140.

Steps to reproduce
==================

1. Create a new image with 'hw_cpu_policy=dedicated', and a new flavor with 'hw:cpu_thread_policy=shared'
2. Boot an instance using this image and flavor

Expected result
===============

The VM should fail to boot, citing a conflict in policies

Actual result
=============

The VM boots.

Additional information
======================

There are two NUMA related properties:

* CPU allocation policy. Values: dedicated(stricter), shared(softer, default)
* CPU thread allocation policy. Values: prefer(softer, default), isolate(medium), require(stricter)

The behaviour in case of conflicts [1]:

CPU allocation policy:

  flavor image result
  -----------------------------------------------------------
  dedicated shared dedicated (stricter, non-default)
  shared dedicated exception

CPU thread allocation policy:

  flavor image result
  ------------------------------------------------------------
  prefer isolate/require prefer (softer, default)
  isolate prefer/requre exception
  require prefer/isolate exception

As you see we have two rather different behaviors for NUMA related properties. If first case we prefer stricter non-default value, in second case we prefer softer default value.

Also the **only** phrase in spec for all this stuff [2] is: "Image property will only be honoured if the flavor does not already have a threads policy set. This ensures the cloud administrator can have absolute control over threads policy if desired."
I'll repeat. This is the *only* phrase about conflicts. Flavor docs [3] says nothing about same image properties and conflict cases. So as user at this point I have no idea what will happen in a case of conflict.

Moreover, in image metadata docs [4] we have brilliant phrase: "Image metadata takes precedence over flavor extra specs. Thus, configuring competing policies causes an exception. By setting a shared policy through image metadata, administrators can prevent users configuring CPU policies in flavors and impacting resource utilization."

So now as user I will say "WTF?". I know that the last example is a docs bug. But even we will replace "image" with "flavour" it wouldn't become a complitly true. There is no information about first lines in each policy conflicts table (at the begining of my message). in those cases we wouldn't get an exception, everything will work fine. But user don't know about it. And user don't know what is 'fine'? Will be used 'shared' policy or 'dedicated'?
We have such horrible docs about two years. The only way to figure out what is going on is to look at the code.

[1] https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1141-L1172
[2] https://specs.openstack.org/openstack/nova-specs/specs/juno/approved/virt-driver-cpu-pinning.html
[3] http://docs.openstack.org/admin-guide/compute-flavors.html
[4] http://docs.openstack.org/admin-guide/compute-cpu-topologies.html