Neutron DVR uses proxy_arp in fip namespaces to respond to arp requests for instance floating ips. In doing so it is susceptible to a random delay up to by default 800ms which is added to the time taken to respond to an arp request that has to be proxied i.e.
# ip netns exec fip-a297543b-9ef9-4bd5-b1ca-e85a726c1726 sysctl net.ipv4.{conf.fg-51f3e07b-2d.proxy_arp,neigh.fg-51f3e07b-2d.proxy_delay}
net.ipv4.conf.fg-51f3e07b-2d.proxy_arp = 1
net.ipv4.neigh.fg-51f3e07b-2d.proxy_delay = 80
The result of this is seen when e.g. you ping a vm fip and the first request takes significantly longer than subsequent requests:
$ ping -c 5 10.5.150.90
PING 10.5.150.90 (10.5.150.90) 56(84) bytes of data.
64 bytes from 10.5.150.90: icmp_seq=1 ttl=60 time=491 ms
64 bytes from 10.5.150.90: icmp_seq=2 ttl=60 time=1.08 ms
64 bytes from 10.5.150.90: icmp_seq=3 ttl=60 time=1.39 ms
64 bytes from 10.5.150.90: icmp_seq=4 ttl=60 time=1.16 ms
64 bytes from 10.5.150.90: icmp_seq=5 ttl=60 time=1.03 ms
--- 10.5.150.90 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4007ms
rtt min/avg/max/mdev = 1.034/99.157/491.134/195.988 ms
To repro again simply delete arp entry for fip from fip ns of source compute host.
By kernel standards this behaviour is by-design when using the default settings but some workloads may be impacted by this initial delay especially e.g. in loaded environments where the arp caches are under strain and hitting gc_thresh limits.
This can alternatively easily be fixed by changing the default vi the charm using the sysctl config option but of course that would not fix existing fip namespaces.