Incorrect port unbind request on instance delete

[AJAY KALAMBUR]

Nova sends a request for port unbind with binding host "" and this can sometimes lead to a problem when nova's cache is different than Neutron's view of instance-port mappings.

Steps
1. Create a neutron port
2. Create a VM and launch instance with this port
3. Shutdown nova compute and neutron agents on compute node where this VM was spawned
4. Unbind port from VM and delete the VM(Notice offline delete is a DB operation as request cant be completed on compute node as it is down)
5. Now create a new VM with same port on a different compute node
6. Bring up nova compute on the first node

After 30 minutes the running deleted instance reap runs and detects a VM in libvirt which is not present in nova database. It cleans up the VM but in the process it unbinds the pre-existing port which is now attached to a completely different VM on a different compute node

Root cause: nova unbind request does not check if port is bound to current instance from Neutron standpoint

[AJAY KALAMBUR (akalambu)]

Tried following fix seemed to work
diff --git a/nova/network/neutronv2/api.py b/nova/network/neutronv2/api.py
index 7192031..c526393 100644
--- a/nova/network/neutronv2/api.py
+++ b/nova/network/neutronv2/api.py
@@ -1170,7 +1170,11 @@ class API(base_api.NetworkAPI):
         ports = set(ports) - ports_to_skip

         # Reset device_id and device_owner for the ports that are skipped
- self._unbind_ports(context, ports_to_skip, neutron)
+ # Only unbind port if neutron sees this port as bound
+ if data.get('ports', []):
+ self._unbind_ports(context, ports_to_skip, neutron)
+ else:
+ LOG.debug("Skipping port unbind as neutron does not see any ports attached")
         # Delete the rest of the ports
         self._delete_ports(neutron, instance, ports, raise_if_fail=True)

[Jay Pipes (jaypipes)]

Marking as Low priority as I don't see this as anything more than a corner case.

[Ian Wells (ijw-ubuntu)]

This is a means to re-use the port (and its allocated address and MAC) without deleting the port and creating a new one with the same address and MAC. Obviously, you can't create a port before deletion with the same address and MAC, but you also can't create the port reliably after deletion because you can lose either address to another port - it's a race condition.

In the original test case, the issue was a host going down cauing a VM failure. The VM doesn't delete promptly on a freshly downed host (and deleting a VM takes an uncertain amount of time in any case), so it's impossible to free the port by conventional means, and replacing the VM requires recycling its address.

[Sean Dague (sdague)]

There are no currently open reviews on this bug, changing
the status back to the previous state and unassigning. If
there are active reviews related to this bug, please include
links in comments.

[Sean Dague (sdague)]

Found open reviews for this bug in gerrit, setting to In Progress.

review: https://review.openstack.org/387687 in branch: master