Create port get quota related DBDeadLock

[LIU Yulong]

ENV:
Neutron stable/queens (12.0.1)
CentOS 7 (3.10.0-514.26.2.el7.x86_64)
Ceph v10.2.9 Jewel

Exception:
2018-06-20 14:21:52.070 140217 ERROR oslo_middleware.catch_errors DBDeadlock: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') [SQL: u'UPDATE quotausages SET dirty=%(dirty)s WHERE quotausages.project_id = %(quotausages_project_id)s AND quotausages.resource = %(quotausages_resource)s'] [parameters: {'quotausages_project_id': u'f4ff15be8de443b78baf21640d93132b', 'dirty': 1, 'quotausages_resource': u'port'}] (Background on this error at: http://sqlalche.me/e/2j85)

API req and resp:
req:
2018-06-20 14:21:35.997 140217 DEBUG neutron.api.v2.base [req-1622426f-193c-439a-ac8e-09f9cd1809e5 03570241e4ea4f2d8e52e48eabc73f8e f4ff15be8de443b78baf21640d93132b - default default] Request body: {u'port': {u'network_id': u'014d37e6-1d99-42de-8023-0859c0721ddc', u'tenant_id': u'f4ff15be8de443b78baf21640d93132b', u'device_id': u'ccdc82d7-cf7e-473d-8721-b09a91a5f10f', u'admin_state_up': True}} prepare_request_body /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:690

500 resp:
2018-06-20 14:21:52.075 140217 INFO neutron.wsgi [req-1622426f-193c-439a-ac8e-09f9cd1809e5 03570241e4ea4f2d8e52e48eabc73f8e f4ff15be8de443b78baf21640d93132b - default default] 10.129.169.147 "POST /v2.0/ports HTTP/1.1" status: 500 len: 399 time: 16.0932069

LOG:
http://paste.openstack.org/show/723982/

How to reproduce:
Use nova boot with --min_count 300 to create 1core1g test VM.

[LIU Yulong (dragon889)]

Similar bug:
Deadlock with quota when deleting port
https://bugs.launchpad.net/neutron/+bug/1771293

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/683128

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/683128
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ab286bcdaccb788ab9df3186e0605e93a9b10bbc
Submitter: Zuul
Branch: master

commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/695351

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/695352

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/695352
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3ec7aed8a367ba2a44fed300b08797e02fc47e06
Submitter: Zuul
Branch: stable/stein

commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/695351
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ff810fcd80af8054763b0751e1309944d39bbc81
Submitter: Zuul
Branch: stable/train

commit ff810fcd80af8054763b0751e1309944d39bbc81
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/698002

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/698002
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=22250e783b194b9d188268c0d301581d73bd430d
Submitter: Zuul
Branch: stable/rocky

commit 22250e783b194b9d188268c0d301581d73bd430d
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/699193

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699193
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2084585b81649b2afa747e24f036d54e5e16e90e
Submitter: Zuul
Branch: stable/queens

commit 2084585b81649b2afa747e24f036d54e5e16e90e
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)
    (cherry picked from commit 22250e783b194b9d188268c0d301581d73bd430d)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/699938

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 13.0.6 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.1 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/699938
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=8e88ae8dc32239a21985d331ec25641f0a4757a5
Submitter: Zuul
Branch: stable/pike

commit 8e88ae8dc32239a21985d331ec25641f0a4757a5
Author: Oleg Bondarev <email address hidden>
Date: Thu Sep 19 16:11:06 2019 +0400

    Set DB retry for quota_enforcement pecan_wsgi hook

    The hook starts a DB transaction and should be covered with
    DB retry decorator.

    For Rocky backport had to import neutron_lib.db for retry_db_errors.

    Closes-Bug: #1777965
    Closes-Bug: #1771293
    Change-Id: I044980a98845edc7b0a02e3323a1e62eb54c10c7
    (cherry picked from commit ab286bcdaccb788ab9df3186e0605e93a9b10bbc)
    (cherry picked from commit 3ec7aed8a367ba2a44fed300b08797e02fc47e06)
    (cherry picked from commit 22250e783b194b9d188268c0d301581d73bd430d)
    (cherry picked from commit 2084585b81649b2afa747e24f036d54e5e16e90e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 16.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/740977

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/740977
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron pike-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron queens-eol release.