[L2][scale issue] ovs-agent has multipe cookies flows (stale flows)

[LIU Yulong]

When subnets or security group ports quantity reach 2000+, there are many stale flows.
Some basic exception procedure:
(1) ovs-agent dump-flows
(2) ovs-agent delete some flows
(3) ovs-agent install new flows (with new cookies)
(4) any exception raise in (2) or (3), such as (bug #1813705)
(5) ovs-agent will do full sync again, then go back to (1)
Finally it will get many stale flows installed, and sometimes cause data plane down.

This is a subproblem of bug #1813703, for more information, please see the summary:
https://bugs.launchpad.net/neutron/+bug/1813703

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/638647

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/638647
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f898ffd71fba4f9b8fd9f4cb851fc3976d72396a
Submitter: Zuul
Branch: master

commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/stein
Review: https://review.openstack.org/648207

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/648217

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/queens
Review: https://review.openstack.org/648219

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/648220

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/649414

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648219
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e4bfc7d50ee94502bead86078a123676bc9c24f9
Submitter: Zuul
Branch: stable/queens

commit e4bfc7d50ee94502bead86078a123676bc9c24f9
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
    (cherry picked from commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648220
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fb84771d1364d9be6fa7d0bce1bc89b2e3541271
Submitter: Zuul
Branch: stable/pike

commit fb84771d1364d9be6fa7d0bce1bc89b2e3541271
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
    (cherry picked from commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648217
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=af67d516a5b39b883fa6fb2fca4673fb7602b292
Submitter: Zuul
Branch: stable/rocky

commit af67d516a5b39b883fa6fb2fca4673fb7602b292
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
    (cherry picked from commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/648207
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7865264aaba615f6e52f5806d844531696186d56
Submitter: Zuul
Branch: stable/stein

commit 7865264aaba615f6e52f5806d844531696186d56
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
    (cherry picked from commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649414
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ea3d844c75541cc2be17865bad6336cd1b8385c4
Submitter: Zuul
Branch: stable/ocata

commit ea3d844c75541cc2be17865bad6336cd1b8385c4
Author: LIU Yulong <email address hidden>
Date: Wed Feb 20 19:46:53 2019 +0800

    Divide-and-conquer local bridge flows beasts

    The dump-flows action will get a very large sets of flow information
    if there are enormous ports or openflow security group rules. For now
    we can meet some known exception during such action, for instance,
    memory issue, timeout issue.
    So after this patch, the cleanup action of the bridge stale flows
    will be done one table by one table. But note, this only supports
    for 'native' OpenFlow interface driver.

    Related-Bug: #1813703
    Related-Bug: #1813712
    Related-Bug: #1813709
    Related-Bug: #1813708

    Change-Id: Ie06d1bebe83ffeaf7130dcbb8ca21e5e59a220fb
    (cherry picked from commit f898ffd71fba4f9b8fd9f4cb851fc3976d72396a)

[Slawek Kaplonski (slaweq)]

Should we maybe consider this bug as fixed now? Or there is still some work to do there?