When update a firewall, we should update the iptables firstly, and then clear the conntrack record, just like the function create_firewall(). Otherwise, the contrack record could be reproduced.

[wujun]

environment: devstack master

When update a firewall, we should update the iptables firstly, and then clear the conntrack record, just like the function create_firewall(). Otherwise, the contrack record could be reproduced.

We can trigger the firewall_update action by:
1.#neutron firewall-update f1 --no-routers
2.vm ping external ip address all the time
3.#neutron firewall-update f1 --router demo-router

We can found that vm still can ping external ip address successfully.

notice:
We should make sure that never stop ping and the interval of ping is small. If it is still not reproduced, we can modify the code to add a "sleep" before the function "_setup_firewall()".

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/471301

[Cao Xuan Hoang (hoangcx)]

I CAN'T reproduce the problem you reported.
The following are steps I tried to reproduce with master (not your patch) in order to verify the problem:
1. Create topology:
openstack network create A100 --project $(openstack project list | grep '\sadmin' | awk '{print $2}')

openstack subnet create --network A100 --subnet-range 100.0.0.0/24 A100

openstack server create --flavor 1 --image cirros-0.3.5-x86_64-disk --nic net-id=$(openstack network list | grep A100 | awk '{print $2}') hoangcx_VM1

openstack router create hoangcx_router

openstack router add subnet $(openstack router list | grep hoangcx_router | awk '{print $2}') $(openstack subnet list | grep A100 | awk '{print $2}')

openstack router set --external-gateway public hoangcx_router

2. Create firewall:
neutron firewall-rule-create --protocol icmp --action deny --name hoangcx_rule

neutron firewall-policy-create --firewall-rules hoangcx_rule hoangcx_policy

neutron firewall-create --router hoangcx_router --name hoangcx_fw hoangcx_policy

3. Access to VM and try to ping to external IP
=> Can't ping (because of FW applied on Router)

4. Update FW (your step 1 in lunchpad) and check ping results

neutron firewall-update hoangcx_fw --no-routers
=> Can ping

5. Update FW again (your step 3) and check ping results

neutron firewall-update hoangcx_fw --router hoangcx_router

=> Can't ping.

Everything looks fine.

So, Please let me know if I'm missing something.

[wujun (wujun)]

When we ping the external ip address in the VM, we should not stop it during the all steps.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/471988

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/471989

[Cao Xuan Hoang (hoangcx)]

Confirmed!

The reason because the interval really less. I picked a breakpoint at [1] and the problem occurred.
Thanks for pointed out the problem.

[1] https://github.com/openstack/neutron-fwaas/blob/master/neutron_fwaas/services/firewall/drivers/linux/iptables_fwaas.py#L155

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/471301
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=6a31bfbb3400fb818e7b2f15fa11337bafed80cd
Submitter: Jenkins
Branch: master

commit 6a31bfbb3400fb818e7b2f15fa11337bafed80cd
Author: wujun <email address hidden>
Date: Tue Jun 6 05:58:32 2017 -0400

    Modify an order between iptables and conntrack when update firewall

    When update a firewall, we should update the iptables firstly,
    and then remove the conntrack record, just like the function
    create_firewall() and create_firewall_group(). Otherwise, the
    contrack record could be reproduced. It will be occurred more
    easily in scenario of large flow, because removing conntrack
    and updating firewall will take some time, and in this interval
    the subsequent flow could be came to reproduced the same
    conntrack record.

    Change-Id: I7bd36964199c6ce7c146f3ef06a693e9c6fe5353
    Closes-bug: #1696093

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-fwaas 11.0.0.0b2 development milestone.

[Ihar Hrachyshka (ihar-hrachyshka)]

For what I understand, scenario here is that even after killing allow rule from firewall, an unfortunate timing may still keep conntrack and connectivity that was allowed by the rule. I consider it a significant breakage of core behaviour of the project and raise the importance to High.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/471988
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=1471eba89d06fd564737e39da9d2fe81fe8aed3a
Submitter: Jenkins
Branch: stable/newton

commit 1471eba89d06fd564737e39da9d2fe81fe8aed3a
Author: wujun <email address hidden>
Date: Tue Jun 6 05:58:32 2017 -0400

    Modify an order between iptables and conntrack when update firewall

    When update a firewall, we should update the iptables firstly,
    and then remove the conntrack record, just like the function
    create_firewall() and create_firewall_group(). Otherwise, the
    contrack record could be reproduced. It will be occurred more
    easily in scenario of large flow, because removing conntrack
    and updating firewall will take some time, and in this interval
    the subsequent flow could be came to reproduced the same
    conntrack record.

    Change-Id: I7bd36964199c6ce7c146f3ef06a693e9c6fe5353
    Closes-bug: #1696093

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/471989
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=8d56f62fb8d5af6831673d2c1d602d4ee2744437
Submitter: Jenkins
Branch: stable/ocata

commit 8d56f62fb8d5af6831673d2c1d602d4ee2744437
Author: wujun <email address hidden>
Date: Tue Jun 6 05:58:32 2017 -0400

    Modify an order between iptables and conntrack when update firewall

    When update a firewall, we should update the iptables firstly,
    and then remove the conntrack record, just like the function
    create_firewall() and create_firewall_group(). Otherwise, the
    contrack record could be reproduced. It will be occurred more
    easily in scenario of large flow, because removing conntrack
    and updating firewall will take some time, and in this interval
    the subsequent flow could be came to reproduced the same
    conntrack record.

    Change-Id: I7bd36964199c6ce7c146f3ef06a693e9c6fe5353
    Closes-bug: #1696093

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-fwaas 9.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-fwaas 10.1.0 release.