Deadlock with quota when deleting port

[Thomas Herve]

Found here: http://logs.openstack.org/38/567238/3/check/heat-functional-convg-mysql-lbaasv2-py35/295509a/logs/screen-q-svc.txt.gz#_May_14_09_25_12_996826

The following query fails:

oslo_db.exception.DBDeadlock: (pymysql.err.InternalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') [SQL: 'UPDATE quotausages SET dirty=%(dirty)s WHERE quotausages.project_id = %(quotausages_project_id)s AND quotausages.resource = %(quotausages_resource)s'] [parameters: {'dirty': 1, 'quotausages_project_id': '9774958ed24f4e28b5d2f5d72863861d', 'quotausages_resource': 'network'}

The incoming DELETE API call fails with a 500.

[LIU Yulong (dragon889)]

I just submitted a new bug with similar exception, but it's during port creating:
https://bugs.launchpad.net/neutron/+bug/1777965

[pandatt (pandatt)]

The artical points out root cause for this issue and possible solution for it. https://stackoverflow.com/questions/2332768/how-to-avoid-mysql-deadlock-found-when-trying-to-get-lock-try-restarting-trans

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/683128

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/683128
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ab286bcdaccb788ab9df3186e0605e93a9b10bbc
Submitter: Zuul
Branch: master

commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/695351

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/695352

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/695352
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3ec7aed8a367ba2a44fed300b08797e02fc47e06
Submitter: Zuul
Branch: stable/stein

commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/695351
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ff810fcd80af8054763b0751e1309944d39bbc81
Submitter: Zuul
Branch: stable/train

commit ff810fcd80af8054763b0751e1309944d39bbc81
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/698002

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698002
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=22250e783b194b9d188268c0d301581d73bd430d
Submitter: Zuul
Branch: stable/rocky

commit 22250e783b194b9d188268c0d301581d73bd430d
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/699193

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699193
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2084585b81649b2afa747e24f036d54e5e16e90e
Submitter: Zuul
Branch: stable/queens

commit 2084585b81649b2afa747e24f036d54e5e16e90e
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)
    (cherry picked from commit 22250e783b194b9d188268c0d301581d73bd430d)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/699938

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.6 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.1 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699938
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8e88ae8dc32239a21985d331ec25641f0a4757a5
Submitter: Zuul
Branch: stable/pike

commit 8e88ae8dc32239a21985d331ec25641f0a4757a5
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)
    (cherry picked from commit 22250e783b194b9d188268c0d301581d73bd430d)
    (cherry picked from commit 2084585b81649b2afa747e24f036d54e5e16e90e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/740977

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/740977
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron pike-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron queens-eol release.