neutron-keepalived-state-change and keeplived cannot be cleanup for those routers which is deleted during l3-agent died

[yao ning]

Pre-conditions:
deploy a 3 controller openstack cluster with stable/rocky branch without using docker, and enable l3 agent ha in configuration files: neutron.conf

l3_ha = true
max_l3_agents_per_router = 3

reproduction steps:
1. create a router and wait for neutron-keepalived-state-change and keepalived becomes to running state. (on control-1)
2. stop the neutron-l3-agent on control-1, systemctl stop neutron-l3-agent
3. delete the router and wait for the process complete so that neutron router show <router_id> report the router is non-existed.
4. start the neutron-l3-agent again on control-1 until neutron agent-list reporting it alive
5. ps -ef | grep <router_id> , you will find neutron-keepalived-state-change and keepalived still there and cannot be terminated anymore until we restart the node.

[Miguel Lavalle (minsel)]

Hi,

Thanks for filing this bug. I have two questions:

1) What happens if you replicate the experiment only with two controllers? Do you see the same behavior

2) Is this something that happens in production? Or was this an experiment with devstack or a test bed?

[Miguel Lavalle (minsel)]

In those words, how much of a real impact does this make? The reason I ask is that the repro steps seem a little contrived: stop the l3 agent and then delete the router

[yao ning (mslovy11022)]

I just find resource leakage in our production environment with a optical fiber issues on control-1, since this is a overlay network, the namespace is isollated networks. Therefore, we don't find a further impaction on our system now. thanks, @Miguel Lavalle (minsel)