L3HA Router in primary state on all nodes

Bug #1899967 reported by Slawek Kaplonski
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Critical
Slawek Kaplonski

Bug Description

It seems that after we merged patch https://review.opendev.org/#/c/748719/ we broke communication between keepalived processes on ha network and because of that HA routers are always master on all nodes.

We found that in tobiko job: https://c5e3229f1913e164c6b4-b80014e5a7a6453c822c4dc8f22159da.ssl.cf2.rackcdn.com/758033/2/check/devstack-tobiko-faults-centos/fb012bd/test_results_01_create_resources_scenario.html

Tags: l3-ha tobiko
Revision history for this message
Slawek Kaplonski (slaweq) wrote :

It is now broken with openvswitch fw driver because in that case filtering is done on br-int level. In case of iptables_hybrid driver it was never an issue because ha_port is always plugged directly to br-int so iptables can't filter packets to that port at all.

Revision history for this message
Slawek Kaplonski (slaweq) wrote :

I think that we will need to create new, "special" SG when ha network is created and apply this SG to all ha ports in tenant. This rule should allows multicast traffic.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/758499

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.opendev.org/758499
Reason: This solution would fix original problem and introduce some new ones.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/759555

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/759679

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/759555
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=14a1ad7009fb20f21bf58accbd339264bebed3b9
Submitter: Zuul
Branch: master

commit 14a1ad7009fb20f21bf58accbd339264bebed3b9
Author: Slawek Kaplonski <email address hidden>
Date: Sat Oct 24 08:27:38 2020 +0000

    Revert "Process ingress multicast traffic for 224.0.0.X separately"

    This reverts commit b8be1a05facff2ba8b484902494ce1663e0aae7c.

    As was reported in bug [1] this patch broke multicast traffic send
    from ports with disabled port security. And that broke L3HA routers
    as keepalived processes couldn't talk to each other.
    During attempt to fix that issue with keepalived we found out another
    corner cases which we may break and in fact to fix them, we would
    effectively revert this change and allow multicast traffic for all
    ports in e.g. networks with ports which have port security and ports
    which don't have port security and are on same node.
    As we also don't really know what other corner cases we may hit going
    further with that, lets revert this patch.
    As a follow up patch I will propose new patch which will document
    differences in handling multicast traffic between iptables and
    openvswitch based firewall drivers.

    [1] https://bugs.launchpad.net/neutron/+bug/1899967

    Change-Id: I37a8b33cf8e16d5bb5dc1966fc2dca6bb619026c
    Closes-Bug: #1899967

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/759679
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d842d0dbf0e1870595eb9d1bf168691832cec1aa
Submitter: Zuul
Branch: master

commit d842d0dbf0e1870595eb9d1bf168691832cec1aa
Author: Slawek Kaplonski <email address hidden>
Date: Mon Oct 26 13:52:31 2020 +0100

    [Docs] Add info about how multicast is treated by fw drivers

    This patch adds info about how multicast traffic is treated by
    openvswitch and iptables based firewall drivers.
    Patch [1] was trying to fix behaviour of OVS based driver to make
    it similar to how iptables drivers works but it introduced bug [2]
    which we wasn't able to fix without basically disabling what [1] did
    for some ports on the compute nodes.
    So based on that we decided to revert [1] - it is done in [3] and to
    document different behaviour between those 2 firewall drivers which is
    done by this patch.

    [1] https://review.opendev.org/#/c/748719/
    [2] https://bugs.launchpad.net/neutron/+bug/1899967
    [3] https://review.opendev.org/#/c/759555/

    Change-Id: If8a56579c62f58befdc57f5916a5763e9fb99531
    Related-Bug: #1899967
    Related-Bug: #1889631

tags: added: neutron-proactive-backport-potential
tags: removed: neutron-proactive-backport-potential
tags: added: tobiko
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.0.0.0rc1

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.