iptables duplicate rule warning on ports with multiple security groups

[Kevin Benton]

If ports are members of multiple security groups, there may be duplicate rules when it comes time to convert them to iptables rules (e.g. both groups have a rule to allow TCP port 80). This results in warnings from the iptables manager detecting duplicate rules that hint that there may be a bug.

For example:

WARNING neutron.agent.linux.iptables_manager [req-944a9996-062b-4588-9536-d5df779da344 - -] Duplicate iptables rule detected. This may indicate a bug in the the iptables rule generation code. Line: -A neutron-openvswi-oe4186b39-0 -j RETURN

This warning resulted from a port that was a member of two security groups that both allowed all EGRESS traffic.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/301029

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/301029
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=142b68f0757ab036d56bc9b4563b7a4481527deb
Submitter: Jenkins
Branch: master

commit 142b68f0757ab036d56bc9b4563b7a4481527deb
Author: Kevin Benton <email address hidden>
Date: Fri Apr 1 01:53:10 2016 -0700

    De-dup user-defined SG rules before iptables call

    A port may be a member of multiple security groups. These
    security groups may have dupilcate rules between them
    (e.g. they both allow all EGRESS traffic). If the iptables
    manager is called with duplicated rules, it emits a warning
    of a possible bug in the rule generation code because it
    doesn't expect there to be duplicated rules.

    This patch fixes this by de-duplicating user-defined security group
    rules before dispatching the calls to the iptables_manager.

    Change-Id: I98dbe60df1bcf68b9922deee63dd0328c4c10dd0
    Closes-Bug: #1565705

[Dr. Jens Harbott (j-harbott)]

Could this patch be backported to Mitaka?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/321773

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 9.0.0.0b1 development milestone.

[Richard Theis (rtheis)]

Is a backport to Liberty also in consideration?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/336152

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/321773
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=98e91541b8d05905b5fcd4a43ab8ced2a0c4c1b1
Submitter: Jenkins
Branch: stable/mitaka

commit 98e91541b8d05905b5fcd4a43ab8ced2a0c4c1b1
Author: Kevin Benton <email address hidden>
Date: Fri Apr 1 01:53:10 2016 -0700

    De-dup user-defined SG rules before iptables call

    A port may be a member of multiple security groups. These
    security groups may have dupilcate rules between them
    (e.g. they both allow all EGRESS traffic). If the iptables
    manager is called with duplicated rules, it emits a warning
    of a possible bug in the rule generation code because it
    doesn't expect there to be duplicated rules.

    This patch fixes this by de-duplicating user-defined security group
    rules before dispatching the calls to the iptables_manager.

    Change-Id: I98dbe60df1bcf68b9922deee63dd0328c4c10dd0
    Closes-Bug: #1565705
    (cherry picked from commit 142b68f0757ab036d56bc9b4563b7a4481527deb)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/336152
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=470d046e91d16204ef64237d89b817c87307b696
Submitter: Jenkins
Branch: stable/liberty

commit 470d046e91d16204ef64237d89b817c87307b696
Author: Kevin Benton <email address hidden>
Date: Fri Apr 1 01:53:10 2016 -0700

    De-dup user-defined SG rules before iptables call

    A port may be a member of multiple security groups. These
    security groups may have dupilcate rules between them
    (e.g. they both allow all EGRESS traffic). If the iptables
    manager is called with duplicated rules, it emits a warning
    of a possible bug in the rule generation code because it
    doesn't expect there to be duplicated rules.

    This patch fixes this by de-duplicating user-defined security group
    rules before dispatching the calls to the iptables_manager.

    Change-Id: I98dbe60df1bcf68b9922deee63dd0328c4c10dd0
    Closes-Bug: #1565705
    (cherry picked from commit 142b68f0757ab036d56bc9b4563b7a4481527deb)

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 7.1.2 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 8.2.0 release.

[sean redmond (sean-redmond1)]

Is the fix in liberty still pending a package update for trusty as the latest in the repo seems to be 7.1.1-0ubuntu1~cloud0

[bhujay kumar (bhatta)]

why same error in ocata ?

WARNING neutron.agent.linux.iptables_manager [-] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xfff

I have deployed using openstack ansible stable/ocata. Checked the code - neutron/agent/linux/iptables_firewall.py in neutron-agent-container and its updated as the fix . Also deleted duplicate rules/groups applied in vm . restarted the neutron server and agent service still the error is coming . Can you give me some hints?