making new allocations for one consumer against multiple resource providers fails with 409

[Chris Dent]

If you PUT some allocations for a new consumer (thus no generation), and those allocations are against more than one resource provider, a 409 failure will happen with:

consumer generation conflict - expected 0 but got None

This because in _new_allocations in handlers/allocation.py we always use the generation provided in the incoming data when we call util.ensure_consumer. This works for the first resource provider but then on the second one the consumer exists, so our generation has to be different now.

One possible fix (already in progress) is to use the generation from new_allocations[0].consumer.generation in subsequent trips round the loop calling _new_allocations.

I guess we must have missed some test cases. I'll make sure to add some when working on this. I found the problem with my placecat stuff.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/577914

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/577915

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/577914
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6b56bba8edab4a324280be4093f7370892a1712c
Submitter: Zuul
Branch: master

commit 6b56bba8edab4a324280be4093f7370892a1712c
Author: Chris Dent <email address hidden>
Date: Mon Jun 25 20:25:16 2018 +0100

    [placement] Demonstrate bug in consumer generation handling

    When PUTting allocations to a non-existent consumer, the code
    expects the consumer generation to be 'null'. When there is only
    one resource provider in the set of allocations sent, this works
    well.

    Unfortunately when there is more than one resource provider the
    loop which calls _new_allocations still uses the 'null' generation
    but now the code expects the generation to be 0, because the
    consumer now exists.

    This patch provides a gabbi test that demonstrates the problem
    by creating a shared disk resource provider and associating it
    as an aggregate with the existing RP_UUID. A GET
    /allocation_candidates is made and a single allocation_request is
    returned including both resource providers. A test with a new
    consumer is marked as an xfail.

    In the next patch the problem will be fixed.

    Change-Id: I1238f82dfc46fd9dae126abfd16ce3dd1506e991
    Related-Bug: #1778576

[OpenStack Infra (hudson-openstack)]

Change abandoned by Chris Dent (<email address hidden>) on branch: master
Review: https://review.openstack.org/577915
Reason: there are other, more complete fixes for this in progress

[Eric Fried (efried)]

This is fixed as part of https://review.openstack.org/#/c/579921/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/579921
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c64198182646e0f264951dc24ed3ef98a297e888
Submitter: Zuul
Branch: master

commit c64198182646e0f264951dc24ed3ef98a297e888
Author: Jay Pipes <email address hidden>
Date: Tue Jul 3 12:48:34 2018 -0400

    placement: delete auto-created consumers on fail

    When we fail to create allocations for new consumers (either when
    issuing a PUT /allocations/{new_consumer_uuid} or a POST /allocations
    where the payload includes a new consumer UUID), we need to ensure that
    we delete the Consumer object and underlying record in the consumers
    table that gets auto-created before calling AllocationList.create_all().

    This auto-created consumer record is what is used to compare things like
    consumer generation in later calls to PUT|POST /allocations, and this
    phantom consumer record was causing confusion when normal retries (for
    things like 409 Conflict due to concurrent provider or inventory
    updates) would be rejected stating that the expected consumer generation
    was 0 and not null (null being the sentinel that indicates the caller is
    expecting the consumer is a new consumer).

    Change-Id: If37ef318bd5482a2d19928002c6f1fa24932946f
    Closes-bug: #1779725
    Closes-bug: #1778576

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 18.0.0.0b3 development milestone.