FWaaS V2: Upgrade Pike->Queen causes error

[German Eichberger]

From our chat:

<jdavis> Jon Davis Hello - I just upgraded to Queens and fwaas_v2 is throwing error: http://paste.openstack.org/raw/688888/
6:46 PM J<jdavis> Jon Davis Everything was working fine in Pike
6:46 PM for attr, position in ATTR_POSITIONS[protocol]: KeyError: 'unknown'
6:47 PM Ideas on where to look?

[chandan dutta chowdhury (chandanc)]

Hello Jon,

After going through the traceback and looking at the code, it seems there might be a conntrack entry in the router namespace for which the kernel could not detect the associated protocol.

Currently the protocols that the FWaaS driver handles are TCP/UDP/ICMP/ICMP6.
The error does not seem to be directly related to the upgrade case, but has been triggered by the firewall update call which tries to cleanup existing contract sessions for a virtual router.

It will be helpful to get a dump of the contract session on your virtual router 9167c6f2-d32a-453d-be52-8628a447fc14 to confirm this

You can run following command to get the dump
sudo ip netns exec <q-router-ns> conntrack -L

I suspect a conntrack entry starting with the unknown protocol in the namespace
e.g.
unknown 2 551 src=0.0.0.0 dst=224.0.0.1 [UNREPLIED] src=224.0.0.1 dst=0.0.0.0 mark=0 use=1

Thanks
Chandan

[chandan dutta chowdhury (chandanc)]

We are discussing the fix for this issue within the team and will update the bug with the details

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/550140

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/550140
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=9b89d4802c113f3eab9114129a4c14175948d2ed
Submitter: Zuul
Branch: master

commit 9b89d4802c113f3eab9114129a4c14175948d2ed
Author: Chandan Dutta Chowdhury <email address hidden>
Date: Tue Mar 6 08:47:35 2018 +0000

    Skip unknown protocols while deleting conntrack

    This patch updates the legacy conntrack driver to skip any
    conntrack entries in the virtual router with an unknown protocol.

    The conntrack driver currently handles sessions for
    TCP/UDP/ICMP/ICMP6 protocols only

    Change-Id: Ic2572086a13ea9c3acc3aee1350b569740aa0d8f
    Closes-Bug: #1753507

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/550822

[chandan dutta chowdhury (chandanc)]

The previous patch https://review.openstack.org/550140 has been reverted (https://review.openstack.org/#/c/550851/) as it was an incorrect fix.

Patch https://review.openstack.org/#/c/550822/ is in progress to fix this bug

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/550822
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=4aa3cec16e341023dea12c7c6af1b9805be6bd81
Submitter: Zuul
Branch: master

commit 4aa3cec16e341023dea12c7c6af1b9805be6bd81
Author: Chandan Dutta Chowdhury <email address hidden>
Date: Thu Mar 8 05:55:22 2018 +0000

    Filter out conntrack entries with unknown protocol

    This patch updates the legacy conntrack driver to skip listing
    (and deleting) any conntrack entries with an unknown protocol.

    The conntrack driver currently handles sessions for
    TCP/UDP/ICMP/ICMP6 protocols only.

    Change-Id: I53439c3f614d1a6ae7b3d679c018065d700316c5
    Closes-Bug: #1753507

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/552390

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-fwaas 13.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552390
Committed: https://git.openstack.org/cgit/openstack/neutron-fwaas/commit/?id=1949733c3f4b1125222637c8eef8f5a4bbee35ea
Submitter: Zuul
Branch: stable/queens

commit 1949733c3f4b1125222637c8eef8f5a4bbee35ea
Author: Chandan Dutta Chowdhury <email address hidden>
Date: Thu Mar 8 05:55:22 2018 +0000

    Filter out conntrack entries with unknown protocol

    This patch updates the legacy conntrack driver to skip listing
    (and deleting) any conntrack entries with an unknown protocol.

    The conntrack driver currently handles sessions for
    TCP/UDP/ICMP/ICMP6 protocols only.

    Change-Id: I53439c3f614d1a6ae7b3d679c018065d700316c5
    Closes-Bug: #1753507
    (cherry picked from commit 4aa3cec16e341023dea12c7c6af1b9805be6bd81)

[Crazik (crazik)]

I can confirm it solves also same issue with FWaaSv1 on Queens, where the same library is used.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-fwaas 12.0.1 release.