duplicate quota entry for project_id/resource causes inconsisten behaviour

[Andrew Karpow]

We experienced in one of our cluster an inconsisten behaviour when setting quota for a tenant (setting security_group to 3 but still returned always 2:

$ curl -g -i -X PUT http://127.0.0.1:9696/v2.0/quotas/3e0fd3f8e9ec449686ef26a16a284265 "X-Auth-Token: $OS_AUTH_TOKEN" -d '{"quota": {"security_group": 3}}'
HTTP/1.1 200 OK
Content-Type: application/json
X-Openstack-Request-Id: req-c6f01da8-1373-4968-b78f-87d7698cde15
Date: Wed, 15 May 2019 14:13:29 GMT
Transfer-Encoding: chunked

{"quota": {"subnet": 1, "network": 1, "floatingip": 22, "l7policy": 11, "subnetpool": 0, "security_group_rule": 110, "listener": 11, "member": 880, "pool": 22, "security_group": 2, "router": 2, "rbac_policy": 5, "port": 550, "loadbalancer": 11, "healthmonitor": 11}}

after some research, we found there is a duplicate entry with same, project_id and resource in the quotas table:

$ SELECT project_id, resource, count(*) as qty FROM quotas GROUP BY project_id, resource HAVING count(*)> 1
            project_id | resource | qty
----------------------------------+----------------+-----
 3e0fd3f8e9ec449686ef26a16a284265 | security_group | 2
(1 row)

deleting one of duplicate entries recovered the correct behaviour. This could be caused by a race-condition or backup leftovers.

I would suggest to add a migration that adds a unique constrain to (project_id, resource), does that sound reasonable?

[Slawek Kaplonski (slaweq)]

IMO this proposal seems reasonable. But we should also remember to check where such race can happened and handle such duplicate entry errors properly in the code.

[kiran pawar (kpdev)]

Possible duplicate of bug https://bugs.launchpad.net/neutron/+bug/1893314 and fixed in https://review.opendev.org/c/openstack/neutron/+/748578/

Andrew Karpow, can you confirm ?

[Andrew Karpow (andyonce)]

Sounds good, we don't upgraded yet to Wallaby, but this commit exactly does what I suggested (unique constrain to (project_id, resource), so pretty sure that's fixing it, thanks