neutron doesnt delete port binding level when deleting an inactive port binding

Bug #1815345 reported by Adrian Chiris
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Adrian Chiris

Bug Description

When performing VM live migration with a normal port (OVS mechanism driver) nova creates an in-active binding on the destination node then activates that binding upon successful migration.

in case of failure to migrate with libvirt, an exception is raised, nova performs a rollback operation for the live migration and instance remains in running state on the source node.

part of the rollback operation is deleting neutron's port binding on the destination node with the following API call:
DELETE /v2.0/ports/​{port_id}​/bindings/{host_id}

this call, for an inactive port binding (which was never activated), does not delete the port's binding level which causes future migration attempts to fail.

Reproduction setup:
- devstack deployment of an all in one and a compute node from master
- OS: FC28
- QEMU hypervisor
- neutron OVS mechanism driver enabled
- perform further configurations to enable live-migration : https://docs.openstack.org/nova/pike/admin/configuring-migrations.html
- block libvirt migration port with iptables on destination node (on my setup i just needed to activate iptables on destination node)

reproduction steps:
http://paste.openstack.org/show/744802/

Revision history for this message
Adrian Chiris (adrian.chiris) wrote :

the following patch proposes a fix to the issue:

https://review.openstack.org/#/c/634276/

summary: - neutron doesnt delete port binding level when deactivating a port
+ neutron doesnt delete port binding level when deleting an inactive port
binding
Changed in neutron:
importance: Undecided → Medium
status: New → Confirmed
status: Confirmed → In Progress
assignee: nobody → Adrian Chiris (adrian.chiris)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/641042

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/634276
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=b197f7c1c4b9c0dd4c58f5c5a4b654dde5596b85
Submitter: Zuul
Branch: master

commit b197f7c1c4b9c0dd4c58f5c5a4b654dde5596b85
Author: Adrian Chiris <email address hidden>
Date: Thu Jan 31 18:51:33 2019 +0200

    Delete port binding level for deleted bindings

    Today, if live migration has failed after an inactive
    binding was created on the destination node but before
    the activation of the created binding, the port's binding level
    for the destination host is not cleared during nova's API call
    to neutron to delete the port binding.

    This causes future attempts to perform live migration
    of the instance to the same host to fail.

    This change removes port binding level object during port binding
    deletion.

    Closes-Bug: #1815345

    Change-Id: Idd55f7d24a2062c08ac8a0dc2243625632d962a5

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 14.0.0.0b3

This issue was fixed in the openstack/neutron 14.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/rocky)

Reviewed: https://review.openstack.org/641042
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=28a7dd12fef78857a0c0f347cf11a63091ea0f5a
Submitter: Zuul
Branch: stable/rocky

commit 28a7dd12fef78857a0c0f347cf11a63091ea0f5a
Author: Adrian Chiris <email address hidden>
Date: Thu Jan 31 18:51:33 2019 +0200

    Delete port binding level for deleted bindings

    Today, if live migration has failed after an inactive
    binding was created on the destination node but before
    the activation of the created binding, the port's binding level
    for the destination host is not cleared during nova's API call
    to neutron to delete the port binding.

    This causes future attempts to perform live migration
    of the instance to the same host to fail.

    This change removes port binding level object during port binding
    deletion.

    Closes-Bug: #1815345

    Change-Id: Idd55f7d24a2062c08ac8a0dc2243625632d962a5
    (cherry picked from commit b197f7c1c4b9c0dd4c58f5c5a4b654dde5596b85)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 13.0.3

This issue was fixed in the openstack/neutron 13.0.3 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.