| 2021-12-13 15:31:47 |
Rodolfo Alonso |
description |
Just in case, this is related to [1].
In [1] we found that we were deleting the reservations always for a specific resource and project, regardless of the date. The solution was to introduce a timeout (with reasonable value of 20 seconds) to filter the existing reservations. Any recent reservation, created by an ongoing request transaction, is keep in the DB.
This bug shows another problem related to situations with very high concurrency. The deletion of the expired reservations cannot be executed at the same time by two or more concurrent transactions. In case this happens, only one transaction will succeed and the others will fail, triggering the DB retry and ending in a DB lock state.
[1]https://bugs.launchpad.net/neutron/+bug/1940311
[2]https://github.com/openstack/neutron/blob/e99d9a9d0697a21ba7ec84465f415f60041f3767/neutron/db/quota/driver_nolock.py#L53-L58 |
Just in case, this is related to [1].
In [1] we found that we were deleting the reservations always for a specific resource and project, regardless of the date. The solution was to introduce a timeout (with reasonable value of 20 seconds) to filter the existing reservations. Any recent reservation, created by an ongoing request transaction, is keep in the DB.
This bug shows another problem related to situations with very high concurrency. The deletion of the expired reservations cannot be executed at the same time by two or more concurrent transactions. In case this happens, only one transaction will succeed and the others will fail, triggering the DB retry and ending in a DB lock state.
Error log: https://paste.opendev.org/show/811637/
[1]https://bugs.launchpad.net/neutron/+bug/1940311
[2]https://github.com/openstack/neutron/blob/e99d9a9d0697a21ba7ec84465f415f60041f3767/neutron/db/quota/driver_nolock.py#L53-L58 |
|
