Normalize the weights instead of using raw values

[OpenStack Infra]

https://review.openstack.org/27160
commit e5ba8494374a1b049eae257fe05b10c5804049ae
Author: Alvaro Lopez Garcia <email address hidden>
Date: Mon Nov 11 17:03:54 2013 +0100

    Normalize the weights instead of using raw values

    The weight system is being used by the scheduler and the cells code.
    Currently this system is using the raw values instead of normalizing them.
    This makes difficult to properly use multipliers for establishing the
    relative importance between two wheighers (one big magnitude could
    shade a smaller one). This change introduces weight normalization so
    that:

    - From an operator point of view we can prioritize the weighers that
      we are applying. The only way to do this is being sure that all the
      weighers will give a value in a known range, so that it is
      not needed to artificially use a huge multiplier to prioritize a
      weigher.

    - From a weigher developer point of view, somebody willing to implement
      one has to care about 1) returning a list of values, 2) setting the
      minimum and maximum values where the weights can range, if they are
      needed and they are significant for the weighing. For a weigher
      developer there are two use cases:

        Case 1: Use of a percentage instead of absolute values (for example, %
        of free RAM). If we compare two nodes focusing on the percentage of free
        ram, the maximum value for the weigher is 100. If we have two nodes one
        with 2048 total/1024 free, and the second one 1024 total/512 free they
        will get both the same weight, since they have the same % of free RAM
        (that is, the 50%).

        Case 2: Use of absolute values. In this case, the maximum of the weigher
        will be the maximum of the values in the list (in the case above, 1024)
        or the maximum value that the magnitude could take (in the case above,
        2048). How this maximum is set, is a decision of the developer. He may
        let the operator choose the behaviour of the weigher though.

    - From the point of view of the scheduler we ensure that it is using
      normalized values, and not leveraging the normalization mechanism to the
      weighers.

    Changes introduced this commit:

    1) it introduces weight normalization so that we can apply multipliers
       easily. All the weights for an object will be normalized between 0.0 and
       1.0 before being sumed up, so that the final weight for a host will be:

        weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...

    2) weights.BaseWeigher has been changed into an ABC so that we enforce
       that all weighers have the expected methods.

    3) weights.BaseWeigher.weigh_objects() does no longer sum up the
       computer weighs to the object, but it rather returns a list that will be
       then normalized and added to the existing weight by BaseWeightHandler

    4) Adapt the existing weighers to the above changes. Namely
        - New 'offset_weight_multiplier' for the cell weigher
          nova.cells.weights.weight_offset.WeightOffsetWeigher
        - Changed the name of the existing multiplier methods.

    5) unittests for all of the introduced changes.

    Implements blueprint normalize-scheduler-weights

    DocImpact: Now weights for an object are normalized before suming them
    up. This means that each weigher will take a maximum value of 1. This
    may have an impact for operators that are using more than one weigher
    (currently there is only one weigher: RAMWeiger) and for operators using
    cells (where we have several weighers). It is needed to review then the
    multipliers used and adjust them properly in case they have been
    modified.

    Docimpact: There is a new configuration option 'offset_weight_multiplier'
    in nova.cells.weights.weight_offset.WeightOffsetWeigher

    Change-Id: I81bf90898d3cb81541f4390596823cc00106eb20

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/87446

[Summer Long (slong-g)]

Added patch https://review.openstack.org/#/c/87446/, but still have tech questions (have asked Alvaro).

[Openstack Gerrit (openstack-gerrit)]

Reviewed: https://review.openstack.org/87446
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=7fcd56c1163cce3aa2d946b87f3ba155067ab72f
Submitter: Jenkins
Branch: master

commit 7fcd56c1163cce3aa2d946b87f3ba155067ab72f
Author: Summer Long <email address hidden>
Date: Tue Apr 15 15:48:06 2014 +1000

    Updated Weighting section

    Put in tables of relevant weighting options for both hosts
    and cells. Updated intro. Added graphic from training
    guides (moved into common and relinked in training guide).

    Closes-bug: #1261521

    Change-Id: Ic9fd323e31016040dbbf18be4969ff829d4edf1e

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-manuals 15.0.0 release.