Arp_responder function has failed since Ocata

[lynncn]

It seems that the code has changed since Ocata.

cat /usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py
--------------------------------------------------------------------------------------
...
# NOTE(haleyb): These neighbour functions live outside the IpNeighCommand
# class since not all callers require it.
def add_neigh_entry(ip_address, mac_address, device, namespace=None, **kwargs):
    """Add a neighbour entry.

    :param ip_address: IP address of entry to add
    :param mac_address: MAC address of entry to add
    :param device: Device name to use in adding entry
    :param namespace: The name of the namespace in which to add the entry
    """
    ip_version = get_ip_version(ip_address)
    privileged.add_neigh_entry(ip_version,
                               ip_address,
                               mac_address,
                               device,
                               namespace,
                               **kwargs)
...
..
--------------------------------------------------------------------------------------

If you set arp_responder=true, you will encounter the following error:

--------------------------------------------------------------------------------------
2018-03-09 12:35:54.590 2317 INFO neutron.agent.securitygroups_rpc [req-13215026-d027-4dff-830c-8c01a3e24b7e - - - - -] Refresh firewall rules
2018-03-09 12:35:54.723 2317 INFO oslo.privsep.daemon [req-ffd165f2-9b17-49f4-9729-f75ab05f4f63 - - - - -] Running privsep helper: ['sudo', 'privsep-helper', '--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/linuxbridge_agent.ini', '--privsep_context', 'neutron.privileged.default', '--privsep_sock_path', '/tmp/tmpOymdtx/privsep.sock']
2018-03-09 12:35:54.737 2317 WARNING oslo.privsep.daemon [-] privsep log: sudo: no tty present and no askpass program specified
2018-03-09 12:35:54.749 2317 CRITICAL oslo.privsep.daemon [req-ffd165f2-9b17-49f4-9729-f75ab05f4f63 - - - - -] privsep helper command exited non-zero (1)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server [req-ffd165f2-9b17-49f4-9729-f75ab05f4f63 - - - - -] Exception during message handling: FailedToDropPrivileges: privsep helper command exited non-zero (1)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_log/helpers.py", line 67, in wrapper
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server return method(*args, **kwargs)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py", line 41, in add_fdb_entries
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server self.fdb_add(context, self._unmarshall_fdb_entries(fdb_entries))
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 895, in fdb_add
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server interface)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 746, in add_fdb_entries
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server self.add_fdb_ip_entry(mac, ip, interface)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 737, in add_fdb_ip_entry
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server ip_lib.add_neigh_entry(ip, mac, interface)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 997, in add_neigh_entry
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server **kwargs)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 204, in _wrap
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server self.start()
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 215, in start
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server channel = daemon.RootwrapClientChannel(context=self)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 327, in __init__
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server raise FailedToDropPrivileges(msg)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server FailedToDropPrivileges: privsep helper command exited non-zero (1)
2018-03-09 12:35:54.750 2317 ERROR oslo_messaging.rpc.server
--------------------------------------------------------------------------------------

[Brian Haley (brian-haley)]

Might just be a missing setup_privsep() call in L2 agent init code.

[Brian Haley (brian-haley)]

Yes, that was the problem. It was actually just fixed in master, but in a larger change. We will have to create a stable backport with just the relevant change.

https://review.openstack.org/#/c/545355/11/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/552111

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/552539

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/552540

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552540
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0f46559557c0942b69503b4dd357d68132da0762
Submitter: Zuul
Branch: stable/ocata

commit 0f46559557c0942b69503b4dd357d68132da0762
Author: Brian Haley <email address hidden>
Date: Mon Mar 12 14:29:48 2018 -0400

    Config privsep in the linuxbridge agent

    In order to enable arp_responder code, the agent must have
    privsep configured or calls to add fdb entries will fail.

    This is a one-file cherry pick of the master change referenced
    below as the rest does not apply.

    Conflicts:
      neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py

    Closes-bug: 1754563

    (cherry picked from commit e0223edf88abc8c1bb8714121a0df8a9997c7a30)

    Change-Id: I80b8c02da5d769dcb7bbf6f9e4659889c7c92650

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552111
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=61aa1fe9512f2a71feffbcce55b82a9fe17651c3
Submitter: Zuul
Branch: stable/queens

commit 61aa1fe9512f2a71feffbcce55b82a9fe17651c3
Author: Brian Haley <email address hidden>
Date: Mon Mar 12 14:29:48 2018 -0400

    Config privsep in the linuxbridge agent

    In order to enable arp_responder code, the agent must have
    privsep configured or calls to add fdb entries will fail.

    This is a one-file cherry pick of the master change referenced
    below as the rest does not apply.

    Closes-bug: 1754563

    (cherry picked from commit e0223edf88abc8c1bb8714121a0df8a9997c7a30)

    Change-Id: I80b8c02da5d769dcb7bbf6f9e4659889c7c92650

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/552539
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=40ac93b806528f49521f6244859d0118c4113aff
Submitter: Zuul
Branch: stable/pike

commit 40ac93b806528f49521f6244859d0118c4113aff
Author: Brian Haley <email address hidden>
Date: Mon Mar 12 14:29:48 2018 -0400

    Config privsep in the linuxbridge agent

    In order to enable arp_responder code, the agent must have
    privsep configured or calls to add fdb entries will fail.

    This is a one-file cherry pick of the master change referenced
    below as the rest does not apply.

    Conflicts:
      neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py

    Closes-bug: 1754563

    (cherry picked from commit e0223edf88abc8c1bb8714121a0df8a9997c7a30)

    Change-Id: I80b8c02da5d769dcb7bbf6f9e4659889c7c92650

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.4 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.6 release.