[CI] "test_live_migration_with_trunk" failing

[Rodolfo Alonso]

Several occurrences during the last day:
* https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_578/864215/2/check/neutron-ovs-tempest-multinode-full/5780784/testr_results.html
* https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_4a0/864051/6/check/neutron-ovs-tempest-multinode-full/4a0fc4f/testr_results.html
* https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d4e/841838/69/check/neutron-ovs-tempest-multinode-full/d4e2c13/testr_results.html

Zuul build list: https://zuul.opendev.org/t/openstack/builds?job_name=neutron-ovs-tempest-multinode-full&skip=0

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Yatin found a new os-vif version was released 1,5 days ago [1]. We bumped to 3.1.0 from 2.8.0.

The patch that could be causing this problem is [2], but that should be confirmed (this is not an statement).

[1]https://github.com/openstack/os-vif/compare/2.8.0...3.1.0
[2]https://review.opendev.org/c/openstack/os-vif/+/841499

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/865026

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/865295

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/865295
Committed: https://opendev.org/openstack/neutron/commit/51827d8e78db4926f3aa347c4b2237a7b210f861
Submitter: "Zuul (22348)"
Branch: master

commit 51827d8e78db4926f3aa347c4b2237a7b210f861
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Tue Nov 15 16:55:56 2022 +0100

    Remove "test_live_migration_with_trunk" execution

    Removed "test_live_migration_with_trunk" test execution from
    "neutron-ovs-tempest-multinode-full" job until LP#1997025 is
    fixed.

    Related-Bug: #1997025
    Change-Id: Id7a0565db9b014e2658002437a2f21facbe47a8f

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/865026

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/865424

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/865424
Committed: https://opendev.org/openstack/neutron/commit/e1d70f9cb0fb7c291672f08151ee03f18da49d74
Submitter: "Zuul (22348)"
Branch: master

commit e1d70f9cb0fb7c291672f08151ee03f18da49d74
Author: Slawek Kaplonski <email address hidden>
Date: Wed Nov 23 13:16:18 2022 +0100

    Disable "test_live_migration_with_trunk" execution from dvr job

    It's follow up for [1] which temporary disabled this test
    on other multinode jobs.

    [1] https://review.opendev.org/c/openstack/neutron/+/865295

    Related-bug: #1997025
    Change-Id: I3cb1e709bb6b4f6acea37fce8bec8bb35b3e84aa

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/837780
Committed: https://opendev.org/openstack/neutron/commit/33de608f04dcc8117eeba63876598dc2ae93013a
Submitter: "Zuul (22348)"
Branch: master

commit 33de608f04dcc8117eeba63876598dc2ae93013a
Author: Miguel Lavalle <email address hidden>
Date: Wed Apr 13 18:00:12 2022 -0500

    Avoid race condition when deleting trunk bridges

    Prior to this change, trunk bridges are created by os-vif but deleted
    by Neutron when the last vif is removed from it. This creates race
    conditions in some use cases, like DPDK with vhostuserclient mode, when
    VMs are rebooted. To avoid these races, Neutron will not delete trunk
    bridges anymore. Their creation and deletion will be os-vif's
    responsiblity. Since [1], Nova uses the os-vif version that contains
    this functionality.

    This patch also changes the trunk status change event. During a live
    migration, when the trunk parent port has been bound to the destination
    host (that means there is only one port binding associated) and the
    status has changed to ACTIVE, the method triggers the subport binding
    to the new host too. This is because there could be a race condition
    between the subport binding, triggered by the OVS agent, and the parent
    port binding, triggered by Nova. If when the OVS agent tries to bind the
    subports, the parent port is still bound to the source host, the subport
    binding remains in the source host too, instead of changing to the
    destination.

    This patch also reverts [2] and [3]. As commented in the previous
    paragraph, this patch fixes the issue reported in LP#1997025. The trunk
    port live migration with ML2/OVS must be fixed with this patch.

    [1]https://review.opendev.org/c/openstack/nova/+/865031
    [2]https://review.opendev.org/c/openstack/neutron/+/865295
    [3]https://review.opendev.org/c/openstack/neutron/+/865424

    Closes-Bug: #1869244
    Closes-Bug: #1997025

    Change-Id: I4e16357f3ff214fcf41e418982806c24088a2665

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 22.0.0.0rc1 release candidate.