Live migration rollback fails if no Neutorn multiple port bindings extension

Bug #1969980 reported by Artom Lifshitz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Artom Lifshitz

Bug Description

This is really a continuation of bug 1888395.

Steps to reproduce
==================
Have Neutron without multiple port bindings extension.
Boot an instance with network interfaces.
Live migrate, and fail the live migration somehow (easy to do in func tests)

Expected result
===============
Rollback happens correctly.

Actual result
=============
Rollback fails with NotImplementedError: Cannot load 'vifs' in the base class

Environment
===========
Reported against stable/wallaby [1], still reproducible in func tests on master.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2077964

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/839091

Revision history for this message
sean mooney (sean-k-mooney) wrote :

https://paste.opendev.org/show/byrCSEnsHYv3sJd8QgV6/
this si the relevent trace back form the bz

Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

Based on the logs this seems like a valid bug.

Changed in nova:
status: New → Triaged
importance: Undecided → Medium
tags: added: live-migration
Changed in nova:
assignee: nobody → Artom Lifshitz (notartom)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/839227

Changed in nova:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.opendev.org/c/openstack/nova/+/839091
Committed: https://opendev.org/openstack/nova/commit/5181bae923bdae2b536affc87d39ddf2c5f7835d
Submitter: "Zuul (22348)"
Branch: master

commit 5181bae923bdae2b536affc87d39ddf2c5f7835d
Author: Artom Lifshitz <email address hidden>
Date: Fri Apr 22 14:21:55 2022 -0400

    Reproduce live migration rollback w/o multi port bindings error

    When the libvirt driver does live migration rollback of an instance
    with network interfaces, it unconditionally refers to
    migrate_data.vifs. These will only be set when Neutron has the
    multiple port bindings extension. We don't handle the case of the
    extension not being present, and currently the rollback will fail with
    a "NotImplementedError: Cannot load 'vifs' in the base class" error.

    Related-bug: 1969980
    Change-Id: Ieef773453ed9f3ced564c1a352fbefbcc6a653ec

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/c/openstack/nova/+/839227
Committed: https://opendev.org/openstack/nova/commit/aa1b0a7ccb1fcf3644784125bfff6950993e1697
Submitter: "Zuul (22348)"
Branch: master

commit aa1b0a7ccb1fcf3644784125bfff6950993e1697
Author: Artom Lifshitz <email address hidden>
Date: Mon Apr 25 10:20:14 2022 -0400

    Fix LM rollback w/o multi port bindings extension

    Previously, the libvirt driver's live migration rollback code would
    unconditionally refer to migrate_data.vifs. This field would only be
    set if the Neutron multiple port bindings extension was in use. When
    it is not in use, the reference would fail with a NotImplementedError.
    This patch wraps the migrate_data.vifs reference in a conditional that
    checks if the vifs field is actually set. This is the only way to do
    it, as in the libvirt driver we do not have access to the network
    API's has_port_binding_extension() helper.

    Closes-bug: 1969980
    Change-Id: I48ca6a77de38e3afaa44630e6ae1fd41d2031ba9

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/yoga)

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/839921

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/nova/+/839922

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/xena)

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/843336

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/nova/+/843337

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/843338

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/843339

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 26.0.0.0rc1

This issue was fixed in the openstack/nova 26.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/nova/+/839921
Committed: https://opendev.org/openstack/nova/commit/2be1570c96697bdb917a086ccdc5a05d3d21e9db
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 2be1570c96697bdb917a086ccdc5a05d3d21e9db
Author: Artom Lifshitz <email address hidden>
Date: Fri Apr 22 14:21:55 2022 -0400

    Reproduce live migration rollback w/o multi port bindings error

    When the libvirt driver does live migration rollback of an instance
    with network interfaces, it unconditionally refers to
    migrate_data.vifs. These will only be set when Neutron has the
    multiple port bindings extension. We don't handle the case of the
    extension not being present, and currently the rollback will fail with
    a "NotImplementedError: Cannot load 'vifs' in the base class" error.

    Related-bug: 1969980
    Change-Id: Ieef773453ed9f3ced564c1a352fbefbcc6a653ec
    (cherry picked from commit 5181bae923bdae2b536affc87d39ddf2c5f7835d)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/nova/+/839922
Committed: https://opendev.org/openstack/nova/commit/5f086d437e8e467ab8c90605c904470c37098227
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 5f086d437e8e467ab8c90605c904470c37098227
Author: Artom Lifshitz <email address hidden>
Date: Mon Apr 25 10:20:14 2022 -0400

    Fix LM rollback w/o multi port bindings extension

    Previously, the libvirt driver's live migration rollback code would
    unconditionally refer to migrate_data.vifs. This field would only be
    set if the Neutron multiple port bindings extension was in use. When
    it is not in use, the reference would fail with a NotImplementedError.
    This patch wraps the migrate_data.vifs reference in a conditional that
    checks if the vifs field is actually set. This is the only way to do
    it, as in the libvirt driver we do not have access to the network
    API's has_port_binding_extension() helper.

    Closes-bug: 1969980
    Change-Id: I48ca6a77de38e3afaa44630e6ae1fd41d2031ba9
    (cherry picked from commit aa1b0a7ccb1fcf3644784125bfff6950993e1697)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 25.2.0

This issue was fixed in the openstack/nova 25.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova xena-eom

This issue was fixed in the openstack/nova xena-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/wallaby)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/843338
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/843339
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.