iptables manager wrongly deletes other agents' rules

[Nell Jerram]

Calico's Felix agent generates iptables chains that intentionally
include rules that the Neutron iptables_manager code considers to be
duplicates - as revealed by logs like these from the DHCP agent:

2017-02-02 18:50:29.482 3376 WARNING neutron.agent.linux.iptables_manager [-] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A felix-to-ebf1bc0b-ba -m mark --mark 0x1000000/0x1000000 -m comment --comment "Profile accepted packet" -j RETURN
2017-02-02 18:50:29.483 3376 WARNING neutron.agent.linux.iptables_manager [-] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A felix-to-3d959cf9-36 -m mark --mark 0x1000000/0x1000000 -m comment --comment "Profile accepted packet" -j RETURN
2017-02-02 18:50:29.483 3376 WARNING neutron.agent.linux.iptables_manager [-] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A felix-from-ebf1bc0b-ba -m mark --mark 0x1000000/0x1000000 -m comment --comment "Profile accepted packet" -j RETURN
2017-02-02 18:50:29.483 3376 WARNING neutron.agent.linux.iptables_manager [-] Duplicate iptables rule detected. This may indicate a bug in the iptables rule generation code. Line: -A felix-from-3d959cf9-36 -m mark --mark 0x1000000/0x1000000 -m comment --comment "Profile accepted packet" -j RETURN

IIUC, iptables_manager then reprograms iptables with these 'duplicates'
removed, and thereby breaks Calico's iptables.

[Armando Migliaccio (armando-migliaccio)]

If I recall correctly this behavior is considered buggy by neutron. Better to check with Kevin before diving in a fix campaign.

http://git.openstack.org/cgit/openstack/neutron/tree/neutron/agent/linux/iptables_manager.py#n620

[Armando Migliaccio (armando-migliaccio)]

That aside, without a level of coordination, it's only natural to expect the neutron agent to wipe unrecognized rules

[Nell Jerram (neil-jerram)]

Hey Armando - I should record that there is already some discussion of this between Kevin and me in https://review.openstack.org/#/c/428764/. (Unfortunately I did things the wrong way round: patch first, and this bug later. But from now on that will hopefully be rectified.)

[Armando Migliaccio (armando-migliaccio)]

Excellent! I appreciate the feedback.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Neil Jerram (<email address hidden>) on branch: master
Review: https://review.openstack.org/428764