I don't think this issue is related to keepalived as keepalived is not used in scenario where those tests are failing.
I deployed similar env as in dvr-multinode scenario job used in gate and I run those tests.
It's very often that one of FIPs configured during test is not pinging. What I found there is that there is no routing to such FIP configured in fip-xxx namespace on host where it should be. Even after restart L3 agent it's not there so I think it's something wrong on server's side instead of some race on L3 agent.
Also when I detached and attached broken FIP to instance again, I saw in L3 agent's logs something like:

Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info [-] L3 agent failure to setup floating IPs: ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "snat-d623a0c3-feed-4d29-8644-5b992d301eca": No such file or directory
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info Traceback (most recent call last):
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 406, in configure_fip_addresses
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     return self.process_floating_ip_addresses(interface_name)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 349, in process_floating_ip_addresses
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     centralized_fip_cidrs = self.get_centralized_fip_cidr_set()
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/l3/dvr_edge_router.py", line 279, in get_centralized_fip_cidr_set
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     return set([addr['cidr'] for addr in device.addr.list()])
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 672, in list
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     self.name, scope, to, filters, ip_version)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 637, in get_devices_with_ip
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     for line in self._run(options, tuple(args)).split('\n'):
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 372, in _run
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     return self._parent._run(options, self.COMMAND, args)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 90, in _run
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     return self._as_root(options, command, args)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 102, in _as_root
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     namespace=namespace)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 110, in _execute
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     log_fail_as_error=self.log_fail_as_error)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 147, in execute
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     returncode=returncode)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info ProcessExecutionError: Exit code: 1; Stdin: ; Stdout: ; Stderr: Cannot open network namespace "snat-d623a0c3-feed-4d29-8644-5b992d301eca": No such file or directory
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info [-] Failed to process floating IPs.: FloatingIpSetupException: L3 agent failure to setup floating IPs
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info Traceback (most recent call last):
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 940, in process_external
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     fip_statuses = self.configure_fip_addresses(interface_name)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info   File "/opt/stack/neutron/neutron/agent/l3/router_info.py", line 411, in configure_fip_addresses
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info     raise n_exc.FloatingIpSetupException(msg)
Nov 16 22:19:20 devstack-ubuntu-allinone neutron-l3-agent[25687]: ERROR neutron.agent.l3.router_info FloatingIpSetupException: L3 agent failure to setup floating IPs


As my agent was configured as "dvr_snat" I tried to change it's mode to "dvr". After restart agent FIP was configured properly and worked fine.

I will continue digging into it.