Gratutuious ARP updates sent by Neutron L3 agent may be ignored by Linux peers

Bug #1732974 reported by Jorge Sorondo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Max Yatsenko
8.0.x
Fix Released
High
Max Yatsenko

Bug Description

An ufortunate scenario in Linux kernel explained in https://patchwork.ozlabs.org/patch/760372/ may result in no gARP being honoured by Linux network peers. To work the kernel bug around, we may want to spread updates more, not to hit default kernel locktime which is 1s.
https://bugs.launchpad.net/neutron/+bug/1690165

Jorge Sorondo (jsorondo)
tags: added: customer-found
tags: added: support
tags: removed: support
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Please provide which MOS version you want this backport to.

Changed in mos:
assignee: nobody → Jorge Sorondo (jsorondo)
status: New → Incomplete
Revision history for this message
Jorge Sorondo (jsorondo) wrote :

mos 8.0

Changed in mos:
status: Incomplete → Confirmed
milestone: none → 9.2-mu-4
assignee: Jorge Sorondo (jsorondo) → MOS Maintenance (mos-maintenance)
importance: Undecided → High
Max Yatsenko (myatsenko)
Changed in mos:
assignee: MOS Maintenance (mos-maintenance) → Max Yatsenko (myatsenko)
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/neutron (9.0/mitaka)

Change abandoned by Max Yatsenko <email address hidden> on branch: 9.0/mitaka
Review: https://review.fuel-infra.org/37232

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/neutron (9.0/mitaka)

Reviewed: https://review.fuel-infra.org/37231
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 5577921157eb974b740148a4e139fd32eb7fc323
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Nov 29 14:42:47 2017

This is a combination of 2 commits:

1st:
Wait 2 seconds between gratuitous ARP updates instead of 1 second

An unfortunate scenario in Linux kernel may end up with no gratuitous
ARP update being processed by network peers, resulting in connectivity
recovery slowdown when moving an IP address between devices.

Change-Id: Iefd0d01d12d06ce6398c4c5634c634991a78bbe9
Closes-Bug: #1732974
(cherry-picked from: 96c5dd6a2b9610d345c8e7df73f25820b5041360)

2nd:
Send both gratuitous ARP REQUESTs and REPLYs

Due to a linux kernel bug [1], network peers risk missing a gratuitous
arp update. This can be safely worked around by enabling arp_accept for
network interfaces that may receive those ARP packets [2]. Sadly, due to
another kernel bug [3], gratuitous ARP packets that we send are not
honoured by arp_accept. If we switch from REPLY to REQUEST packets, then
all recent kernels should enforce ARP table entry override on gARP
received.

REPLYs are left for backwards compatibility in case some network peers
honor REPLYs and not REQUESTs.

[1] https://patchwork.ozlabs.org/patch/760372/
[2] https://bugs.launchpad.net/fuel/+bug/1456272
[3] https://patchwork.ozlabs.org/patch/760373/

Change-Id: Iacd01875bf1299ff68fb5d1009d72088270320bb
Closes-Bug: #1732974
(cherry-picked from: 82831b9d8d4bd61f90610df9eca8c7f6e447f8d8)

Revision history for this message
Max Yatsenko (myatsenko) wrote :

Path for "openstack-ci/fuel-8.0/liberty" branch was merged:
https://review.fuel-infra.org/#/c/37233

Changed in mos:
status: Confirmed → Fix Committed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/neutron (openstack-ci/fuel-8.0/liberty)

Reviewed: https://review.fuel-infra.org/37233
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: d197e3f75ffda33538020ff3b54abf930e8824e2
Author: Ihar Hrachyshka <email address hidden>
Date: Wed Nov 29 17:43:56 2017

This is a combination of 2 commits.

1st:
Wait 2 seconds between gratuitous ARP updates instead of 1 second

An unfortunate scenario in Linux kernel may end up with no gratuitous
ARP update being processed by network peers, resulting in connectivity
recovery slowdown when moving an IP address between devices.

Change-Id: Iefd0d01d12d06ce6398c4c5634c634991a78bbe9
Closes-Bug: #1732974
(cherry-picked from: 96c5dd6a2b9610d345c8e7df73f25820b5041360)

2nd:
Send both gratuitous ARP REQUESTs and REPLYs

Due to a linux kernel bug [1], network peers risk missing a gratuitous
arp update. This can be safely worked around by enabling arp_accept for
network interfaces that may receive those ARP packets [2]. Sadly, due to
another kernel bug [3], gratuitous ARP packets that we send are not
honoured by arp_accept. If we switch from REPLY to REQUEST packets, then
all recent kernels should enforce ARP table entry override on gARP
received.

REPLYs are left for backwards compatibility in case some network peers
honor REPLYs and not REQUESTs.

[1] https://patchwork.ozlabs.org/patch/760372/
[2] https://bugs.launchpad.net/fuel/+bug/1456272
[3] https://patchwork.ozlabs.org/patch/760373/

Change-Id: Iacd01875bf1299ff68fb5d1009d72088270320bb
Closes-Bug: #1732974
(cherry-picked from: 82831b9d8d4bd61f90610df9eca8c7f6e447f8d8)

Changed in mos:
status: Fix Committed → Fix Released
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/neutron (mcp/1.0/mitaka)

Fix proposed to branch: mcp/1.0/mitaka
Change author: Ihar Hrachyshka <email address hidden>
Review: https://review.fuel-infra.org/38063

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.