[L3] add abilitiy for iptables_manager to ensure rule was added only once

[LIU Yulong]

iptables_manager should have abilitiy to ensure rule was added only once. In function [1], it just adds the new rule to the cache list no matter if it is duplicated. And finally, warning LOG [2] will be raised. Sometimes, there will have multiple threads to add rule for one same resource, it may be not easy for users to ensure that their rule generation code was run as expected. So rule will be duplicated in cache. And during the removal procedure, cache has duplicated rules, remove one then there still has same rule remained. As a result, the linux netfilter rule may have nothing changed after user's removal action.

[1] https://github.com/openstack/neutron/blob/master/neutron/agent/linux/iptables_manager.py#L205-L225
[2] https://github.com/openstack/neutron/blob/master/neutron/agent/linux/iptables_manager.py#L718-L725

[LIU Yulong (dragon889)]

Upstream log search:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Duplicate%20iptables%5C%22%20and%20message%3A%20%5C%22may%20indicate%20a%20bug%20in%20the%20iptables%5C%22

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/684237

[Brian Haley (brian-haley)]

Is there a code path adding a rule twice? The log message was meant to try and find that out.

Luckily the OVSFW is the default now too :)

[LIU Yulong (dragon889)]

@Brian,

This can be one example from upstream:
https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_601/683921/2/check/ironic-tempest-dsvm-ironic-inspector-queens/601000a/controller/logs/screen-neutron-agent.txt.gz

Actually most of this was coming from local feature development, we want to add some iptables rule for resources, but the behavior is not easy to manipulate, mostly unexpected. And finally we found this.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/730824

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/730825

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: stable/queens
Review: https://review.opendev.org/730825
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Slawek Kaplonski (slaweq)]

This bug has had a related patch abandoned and has been automatically un-assigned due to inactivity. Please re-assign yourself if you are continuing work or adjust the state as appropriate if it is no longer valid.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: stable/rocky
Review: https://review.opendev.org/730824
Reason: This review is > 4 weeks without comment, and failed Zuul jobs the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "liuyulong <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/684237
Reason: Restore if someday we want this.

[Brian Haley (brian-haley)]

Since the patch on master was abandoned manually I am going to close this.