instances getting multiple IPs after "juju remove-machine --force" on a compute node

[Junien F]

Hi,

Yesterday, I found out that a node what not properly configured in MAAS (let's call it "nodeA"), so I wanted to redeploy it. I freed it with "juju remove-machine --force", fixed its configuration in MAAS, and redeployed nova-compute on it.

Starting then, any nova instance created had 2 IPs, even if I specified only one "--nic net-id=<id>". These 2 IPs were each backed by a neutron port (as expected), and each time, one of the 2 ports was bound on nodeA.

I found some errors in neutron-api :

2018-03-06 13:33:07.067 5041 ERROR neutron.plugins.ml2.rpc [req-14f17ae0-8c0a-4394-8097-07abd945c14a - - - - -] Failed to update device 951bba80-bc03-4667-a2a5-4b74e85a618e down: AgentNotFoundByTypeHost: Agent with agent_type=L3 agent and host=nodeA could not be found

L3 agents are only on neutron-gateway units - I don't understand why neutron would try to find an L3 agent on a compute node. I tried restarting all openstack services on the nova-cloud-controller units, the neutron-api units, and nodeA, but that didn't help.

I then found bug 1737917, which not exactly what I have since we're not using linuxbridge, but it does mention l2pop.

When disabling l2pop (juju config neutron-api l2-population=False), the problem disappeared. When re-enabling it, the problem didn't come back.

Environment :
- cloud running on xenial, pike, and 17.11 charms
- neutron is GRE + ovs
- infra services are running in LXDs on dedicated metals, 2 units for each
- nova-compute was running on an arm64 machine (probably not relevant)

Thanks