Some candidates are missing when multiple sharing provider have multiple shared resources

[Tetsuro Nakamura]

Some candidates are missing when multiple sharing provider have multiple shared resources

Description
===========

There can be legitimately distinct allocation requests with the same combination of providers.
But placement filter out them if the combination of the providers in play is the same combination with that we already have tin he allocation requests

Example
=========

CN(compute node):
    - VCPU: 16

SHR1: a sharing provider in the same aggregate with CN)
    - CUSTOM_RC1: 4
    - CUSTOM_RC2: 8

SHR2(sharing provider2):
    - CUSTOM_RC1: 6
    - CUSTOM_RC2: 3

We ask for VCPU:1,CUSTOM_RC1:1,CUSTOM_RC2:1.

Expected: we should get the following two allocation requests:

[(rp=CN, resources=[VCPU: 1]),
 (rp=SHR1, resources=[CUSTOM_RC1: 1]),
 (rp=SHR2, resources=[CUSTOM_RC2: 1])]

[(rp=CN, resources=[VCPU: 1]),
 (rp=SHR1, resources=[CUSTOM_RC2: 1]),
 (rp=SHR2, resources=[CUSTOM_RC1: 1]),]

Actual: We get only one of those candidates, because the combination of the rps is the same: CN1, SHR1, and SHR2.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/569637

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/569720

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/569637
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1dcd878d18eb8c2df0e6e5853b6771eba6791cee
Submitter: Zuul
Branch: master

commit 1dcd878d18eb8c2df0e6e5853b6771eba6791cee
Author: Tetsuro Nakamura <email address hidden>
Date: Sun May 20 20:10:49 2018 +0900

    Expand tests for multiple shared resources case

    There was a test to validate we get all the possible combinations
    of sharing providers if they have the requested resources.

    However, this test validates the case where one sharing provider
    has single shared resource class. This patch expand the test case
    to validate that we still get all the possible combinations for
    the case where all the sharing providers have multiple shared
    resource classes.

    Change-Id: I89a970362cbfdd25840901f4635a9e0bffb2c94f
    Related-Bug: #1772243

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/569720
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1f37b16e7bfc3ccf76d7f13c595068e2c2ffcc87
Submitter: Zuul
Branch: master

commit 1f37b16e7bfc3ccf76d7f13c595068e2c2ffcc87
Author: Tetsuro Nakamura <email address hidden>
Date: Mon May 21 04:19:12 2018 +0900

    Use list instead of set for duplicate check

    There can be legitimately distinct allocation requests with the same
    combination of providers. For example, in a case where where both
    SHR1 and SHR2 have both CUSTOM_RC1 and CUSTOM_RC2, the following two
    allocation requests are different:

     AllocationRequest(anchor=CN, resource_requests=[
         AllocationRequestResource(rp=CN, resources=[VCPU: 1]),
         AllocationRequestResource(rp=SHR1, resources=[CUSTOM_RC1: 1]),
         AllocationRequestResource(rp=SHR2, resources=[CUSTOM_RC2: 1]),
     ])

     AllocationRequest(anchor=CN, resource_requests=[
         AllocationRequestResource(rp=CN, resources=[VCPU: 1]),
         AllocationRequestResource(rp=SHR2, resources=[CUSTOM_RC1: 1]),
         AllocationRequestResource(rp=SHR1, resources=[CUSTOM_RC2: 1]),
     ])

    But getting allocation candidates, placement had been filtering out
    one of the two assuming them as the same allocation requests because
    the combination of the providers in play was the same.

    This was because placement filtered duplicate candidates using a set
    of provider ids in play while we can't consider resource information
    with the set. This patch fixes the bug by using the list instead of
    the set. This is because we can put resource information in the order
    of that list. Namely, in the example above, we can't detect
    "set([CN, SHR1, SHR2])" is different as "set([CN, SHR2, SHR1])", but
    we can detect [CN, SHR1, SHR2] is different from [CN, SHR2, SHR1].

    Change-Id: I90627dfe85661118c4ec2571fc32ffca99dd73d2
    Blueprint: nested-resource-providers-allocation-candidates
    Closes-Bug: #1772243

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 18.0.0.0b2 development milestone.