Wrong mac address of ARP entry setting for allowed_address_pairs in DVR router

[KyoLee]

When we set a pair of IP and MAC for allowed_address_pairs to a neutron port, Neutron should notify l3-agent to update permanent ARP entry for this IP and MAC in DVR router namespace. But we found out that the MAC address of ARP entry was wrong, it was set to the MAC address of neutron port. It will cause the DVR router to respond the wrong destination MAC address.

Openstack Stein (explicitly_egress_direct is set to true)

Steps to reproduce:
1. Create a neutron port.

The ARP table in DVR router namespace:
[~] # ip netns exec qrouter-acf46b48-73db-494e-95e7-76b0de3efdf8 ip neigh
192.163.99.2 dev qr-50857d66-3f lladdr fa:16:3e:38:a5:33 PERMANENT
192.163.99.159 dev qr-50857d66-3f lladdr 56:6f:a6:bf:00:02 PERMANENT

At this time, VM can ping qrouter through network interface.

2. Set a IP/MAC pair to neutron port. (This IP/MAC are different from the neutron port of IP/MAC)

neutron port-update ce269425-4b3b-472b-a40d-de80fb44b875 --allowed-address-pairs type=dict list=true mac_address=56:6f:a6:bf:00:03,ip_address=192.163.99.160

The ARP table in DVR router namespace:
[~] # ip netns exec qrouter-acf46b48-73db-494e-95e7-76b0de3efdf8 ip neigh
192.163.99.160 dev qr-50857d66-3f lladdr 56:6f:a6:bf:00:02 PERMANENT
192.163.99.2 dev qr-50857d66-3f lladdr fa:16:3e:38:a5:33 PERMANENT
192.163.99.159 dev qr-50857d66-3f lladdr 56:6f:a6:bf:00:02 PERMANENT

As you can see, the first entry of ARP table, the MAC address was set to the same as MAC address of neutron port. If we modify IP and MAC for VM network interface according to allowed_address_pairs, and then we run ping command from VM to qrouter, it will fail.

We monitor the VM's tap interface, it shows:
[~] # tcpdump -nei vnet0 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:00:47.267560 56:6f:a6:bf:00:04 > fa:16:3e:d3:9f:5a, ethertype IPv4 (0x0800), length 98: 192.163.99.160 > 192.163.99.1: ICMP echo request, id 5317, seq 1, length 64
17:00:47.267929 fa:16:3e:d3:9f:5a > 56:6f:a6:bf:00:02, ethertype IPv4 (0x0800), length 98: 192.163.99.1 > 192.163.99.160: ICMP echo reply, id 5317, seq 1, length 64

The qrouter responds ICMP reply with neutron port MAC address instead of allowed_address_pairs MAC address.

Reference code:
/usr/lib/python2.7/site-packages/neutron/db/l3_dvr_db.py

   1096 def update_arp_entry_for_dvr_service_port(self, context, port_dict):
   1097 """Notify L3 agents of ARP table entry for dvr service port.
   1098
   1099 When a dvr service port goes up, look for the DVR router on
   1100 the port's subnet, and send the ARP details to all
   1101 L3 agents hosting the router to add it.
   1102 If there are any allowed_address_pairs associated with the port
   1103 those fixed_ips should also be updated in the ARP table.
   1104 """
   1105 fixed_ips = port_dict['fixed_ips']
   1106 if not fixed_ips:
   1107 return
   1108 allowed_address_pair_fixed_ips = (
   1109 self._get_allowed_address_pair_fixed_ips(context, port_dict))
   1110 changed_fixed_ips = fixed_ips + allowed_address_pair_fixed_ips
   1111 for fixed_ip in changed_fixed_ips:
   1112 self._generate_arp_table_and_notify_agent(
   1113 context, fixed_ip, port_dict['mac_address'],
   1114 self.l3_rpc_notifier.add_arp_entry)

The problem is in line 1113, it always set neutron port's MAC to ARP entry instead of allowed_address_pairs MAC address.

The expected result is that the qrouter should respond ICMP reply with allowed_address_pairs MAC address

[Slawek Kaplonski (slaweq)]

I don't have stein deployment now but I can't reproduce that same issue on neutron deployed from master branch.
And I think that this was fixed by https://review.opendev.org/#/c/737957/
For now I'm closing this bug but if You can reproduce that issue on newer versions, feel free to reopen it.

[KyoLee (kyolee)]

Hi sir:
Is it possible to patch this to stein branch?

Thanks.