This change adds a fast retry loop around
AllocationList._set_allocations if a resource provider generation
conflict happens. It turns out that under high concurrency of allocation
claims being made on the same resource provider conflicts can be quite
common and client side retries are insufficient.
Because both consumer generation and resource provider generations had
raised the same exception there was no way to distinguish between the
two so a child of ConcurrentUpdateDetected has been created as
ResourceProviderConcurrentUpdateDetected. In the future this will allow
us to send different error codes to the client as well, but that change
is not done here.
When the conflict is detected, all the resource providers in the
AllocationList are reloaded and the list objects refreshed.
Logging is provided to indicate:
* at debug that a retry is going to happen
* at warning that all the retries failed and the client is going to
see the conflict
The tests for this are a bit funky: Some mocks are used to cause the
conflicts, then the real actions after a couple of iterations.
This was backported from 72e4c4c8d7fb146862b899337626485dad10f15b with
conflicts because exceptions, tests and object files were moved to
new locations with Rocky and the AllocationList.create_all method was
renamed to the more accurate replace_all. Prior to Rocky there were
no Consumer objects and fewer helper methods in functional tests,
so the test is adjusted accordingly.
Change-Id: Id614d609fc8f3ed2d2ff29a2b52143f53b3b1b9a
Closes-Bug: #1719933
(cherry picked from commit 72e4c4c8d7fb146862b899337626485dad10f15b)
Reviewed: https:/ /review. openstack. org/588569 /git.openstack. org/cgit/ openstack/ nova/commit/ ?id=66a47b76230 35cff38a12fdace 9a325bbfdd9a14
Committed: https:/
Submitter: Zuul
Branch: stable/queens
commit 66a47b7623035cf f38a12fdace9a32 5bbfdd9a14
Author: Chris Dent <email address hidden>
Date: Thu Jul 26 12:00:14 2018 +0100
[placement] Retry allocation writes server side
This change adds a fast retry loop around ist._set_ allocations if a resource provider generation
AllocationL
conflict happens. It turns out that under high concurrency of allocation
claims being made on the same resource provider conflicts can be quite
common and client side retries are insufficient.
Because both consumer generation and resource provider generations had eDetected has been created as viderConcurrent UpdateDetected. In the future this will allow
raised the same exception there was no way to distinguish between the
two so a child of ConcurrentUpdat
ResourcePro
us to send different error codes to the client as well, but that change
is not done here.
When the conflict is detected, all the resource providers in the
AllocationList are reloaded and the list objects refreshed.
Logging is provided to indicate:
* at debug that a retry is going to happen
* at warning that all the retries failed and the client is going to
see the conflict
The tests for this are a bit funky: Some mocks are used to cause the
conflicts, then the real actions after a couple of iterations.
This was backported from 72e4c4c8d7fb146 862b89933762648 5dad10f15b with create_ all method was
conflicts because exceptions, tests and object files were moved to
new locations with Rocky and the AllocationList.
renamed to the more accurate replace_all. Prior to Rocky there were
no Consumer objects and fewer helper methods in functional tests,
so the test is adjusted accordingly.
Change-Id: Id614d609fc8f3e d2d2ff29a2b5214 3f53b3b1b9a 862b89933762648 5dad10f15b)
Closes-Bug: #1719933
(cherry picked from commit 72e4c4c8d7fb146