packet loss during backup L3 HA agent restart

Bug #1771841 reported by Slawek Kaplonski on 2018-05-17
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
High
Slawek Kaplonski

Bug Description

When backup L3 agent in HA deployment is restarted, there is 20-30 seconds of data plane connectivity break to FloatingIP.
It looks that during start of L3 agent, it first enable IPv6 forwarding: https://github.com/openstack/neutron/blob/master/neutron/agent/l3/namespaces.py#L106 and then disable it for non master node: https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L419
That cause send of MLDv2 packet:

08:38:29.815180 Out fa:16:3e:9e:de:0e ethertype IPv6 (0x86dd), length 112: :: > ff02::16: HBH ICMP6, multicast listener report v2, 2 group record(s), length 48

and that breaks somehow connectivity as packets are sent to backup node instead of master one.

tags: added: queens-backport-potential

Related fix proposed to branch: master
Review: https://review.openstack.org/569424

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.openstack.org/569289

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.openstack.org/569424

Reviewed: https://review.openstack.org/569083
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3e9e2a5b4b2654864bb0e2a7d9b841eb03240147
Submitter: Zuul
Branch: master

commit 3e9e2a5b4b2654864bb0e2a7d9b841eb03240147
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 17 12:16:52 2018 +0200

    Disable IPv6 forwarding by default on HA routers

    In case of HA routers IPv6 forwarding is not disabled by default and
    then enabled only on master node.
    Before this patch it was done in opposite way, so forwarding was
    enabled by default and then disabled on backup nodes.
    When forwarding was enabled/disabled for qg- port, MLDv2 packets are
    sent and that might lead to temportary packets loss as packets to
    FIP were sent to this backup node instead of master one.

    Related-Bug: #1771841

    Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6

Reviewed: https://review.openstack.org/571984
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=2ed6e4d7ab253f23fd81d84b127139b9deb438fd
Submitter: Zuul
Branch: stable/queens

commit 2ed6e4d7ab253f23fd81d84b127139b9deb438fd
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 17 12:16:52 2018 +0200

    Disable IPv6 forwarding by default on HA routers

    In case of HA routers IPv6 forwarding is not disabled by default and
    then enabled only on master node.
    Before this patch it was done in opposite way, so forwarding was
    enabled by default and then disabled on backup nodes.
    When forwarding was enabled/disabled for qg- port, MLDv2 packets are
    sent and that might lead to temportary packets loss as packets to
    FIP were sent to this backup node instead of master one.

    Related-Bug: #1771841

    Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6
    (cherry picked from commit 3e9e2a5b4b2654864bb0e2a7d9b841eb03240147)

tags: added: in-stable-queens
tags: added: neutron-proactive-backport-potential

Reviewed: https://review.openstack.org/572055
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ba9fd196a255170ae48149447a99e3d872d49976
Submitter: Zuul
Branch: stable/pike

commit ba9fd196a255170ae48149447a99e3d872d49976
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 17 12:16:52 2018 +0200

    Disable IPv6 forwarding by default on HA routers

    In case of HA routers IPv6 forwarding is not disabled by default and
    then enabled only on master node.
    Before this patch it was done in opposite way, so forwarding was
    enabled by default and then disabled on backup nodes.
    When forwarding was enabled/disabled for qg- port, MLDv2 packets are
    sent and that might lead to temportary packets loss as packets to
    FIP were sent to this backup node instead of master one.

    Related-Bug: #1771841

    Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6
    (cherry picked from commit 3e9e2a5b4b2654864bb0e2a7d9b841eb03240147)

tags: added: in-stable-pike
Changed in neutron:
importance: Undecided → High

Reviewed: https://review.openstack.org/577396
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5e47f30572b7b9224a6ac6df59f0f1b6794c935f
Submitter: Zuul
Branch: stable/ocata

commit 5e47f30572b7b9224a6ac6df59f0f1b6794c935f
Author: Slawek Kaplonski <email address hidden>
Date: Thu May 17 12:16:52 2018 +0200

    Disable IPv6 forwarding by default on HA routers

    In case of HA routers IPv6 forwarding is not disabled by default and
    then enabled only on master node.
    Before this patch it was done in opposite way, so forwarding was
    enabled by default and then disabled on backup nodes.
    When forwarding was enabled/disabled for qg- port, MLDv2 packets are
    sent and that might lead to temportary packets loss as packets to
    FIP were sent to this backup node instead of master one.

    Related-Bug: #1771841

    Change-Id: Ia6b772e91c1f94612ca29d7082eca999372e60d6
    (cherry picked from commit 3e9e2a5b4b2654864bb0e2a7d9b841eb03240147)

tags: added: in-stable-ocata

Change abandoned by Slawek Kaplonski (<email address hidden>) on branch: master
Review: https://review.openstack.org/569458
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers