arping for floating IPs fail on newer kernels

[Brian Haley]

The code to send gratuitous ARPs changed in Liberty to be simpler because we started setting the sysctl net.ipv4.ip_nonlocal_bind to 1 in the root namespace. It seems like in newer kernels (3.19 or so) that this sysctl attribute was added to the namespaces, so now that arping call fails because we are only enabling non-local binds in the root namespace.

This is an example when run by hand:

$ sudo ip netns exec fip-311e3d4a-00ec-46cc-9928-dbc1a2fe3f9a arping -A -I fg-bb6b6721-78 -c 3 -w 4.5 172.18.128.7
bind: Cannot assign requested address

Failing to get that ARP out can affect connectivity to the floating IP.

In order to support either kernel, the code should change to try setting it in the namespace, and if it fails, then set it in the root namespace.

This is backport potential to stable/liberty.

[Ryan Moats (rmoats)]

To flesh out Brian's description, one can check whether this sysctl attribute is around via something like the following:

administrator@mn-compute-1:/opt/stack/logs$ sysctl net.ipv4.ip_nonlocal_bind
net.ipv4.ip_nonlocal_bind = 1
administrator@mn-compute-1:/opt/stack/logs$ ip netns list
fip-311e3d4a-00ec-46cc-9928-dbc1a2fe3f9a
qrouter-f5519abb-0a8b-4598-bdde-9f20c1050011
administrator@mn-compute-1:/opt/stack/logs$ sudo ip netns exec fip-311e3d4a-00ec-46cc-9928-dbc1a2fe3f9a sysctl net.ipv4.ip_nonlocal_bind
net.ipv4.ip_nonlocal_bind = 0
administrator@mn-compute-1:/opt/stack/logs$

[Brian Haley (brian-haley)]

BTW, this is a link to the kernel change that made ip_nonlocal_bind namespace-specific:

http://lists.openwall.net/netdev/2014/09/05/67

So it fits in the 3.19 time frame as we expected.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/236145
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=24acc41b9314729a0fbbc88b1f8cb19a331a3dc7
Submitter: Jenkins
Branch: master

commit 24acc41b9314729a0fbbc88b1f8cb19a331a3dc7
Author: Brian Haley <email address hidden>
Date: Fri Oct 16 18:02:16 2015 -0400

    Set ip_nonlocal_bind in namespace if it exists

    Somewhere in the 3.19 kernel timeframe ip_nonlocal_bind was
    changed to be a per-namespace attribute. To be backwards
    compatible we need to try that first, then fall-back to
    setting the one in the root namespace if it fails.

    Closes-Bug: #1507078
    Change-Id: I018e800bc8d4e85d067aaae865c9b04bf030c980

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/238173

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/kilo
Review: https://review.openstack.org/238236

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/238236
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fc34dbc44e1d65fcad10d94f894bd97153e0f719
Submitter: Jenkins
Branch: stable/kilo

commit fc34dbc44e1d65fcad10d94f894bd97153e0f719
Author: Brian Haley <email address hidden>
Date: Fri Oct 16 18:02:16 2015 -0400

    Set ip_nonlocal_bind in namespace if it exists

    Somewhere in the 3.19 kernel timeframe ip_nonlocal_bind was
    changed to be a per-namespace attribute. To be backwards
    compatible we need to try that first, then fall-back to
    setting the one in the root namespace if it fails.

    Closes-Bug: #1507078
    Change-Id: I018e800bc8d4e85d067aaae865c9b04bf030c980
    (cherry picked from commit 24acc41b9314729a0fbbc88b1f8cb19a331a3dc7)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/238173
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4428b291f3ffe81500b1dd3d75ed71fd615d19fa
Submitter: Jenkins
Branch: stable/liberty

commit 4428b291f3ffe81500b1dd3d75ed71fd615d19fa
Author: Brian Haley <email address hidden>
Date: Fri Oct 16 18:02:16 2015 -0400

    Set ip_nonlocal_bind in namespace if it exists

    Somewhere in the 3.19 kernel timeframe ip_nonlocal_bind was
    changed to be a per-namespace attribute. To be backwards
    compatible we need to try that first, then fall-back to
    setting the one in the root namespace if it fails.

    Closes-Bug: #1507078
    Change-Id: I018e800bc8d4e85d067aaae865c9b04bf030c980
    (cherry picked from commit 24acc41b9314729a0fbbc88b1f8cb19a331a3dc7)

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/neutron 8.0.0.0b1 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/neutron 7.0.1 release.