Sporadic metadata issues when creating OVN migration workload

Bug #1947547 reported by Roman Safronov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Unassigned

Bug Description

When starting OVN migration workload, some of the workload VMs (usually only one of them) is not accessible via ssh. As can be seen in console logs the VM is not able to retrieve metadata.
This is basically due to bug https://bugs.launchpad.net/neutron/+bug/1813787 - nova
starts vm faster then local dvr router is ready thus during the vm's boot
process when vm asks for metadata, there is (yet) no haproxy ready to process
those requests.
If I create router and attach subnet before starting VMs the issue does not happen. So it seems like the order of the workload creation should be changed.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/814357

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → Medium
tags: added: ovn
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/814357
Committed: https://opendev.org/openstack/neutron/commit/d49ce1652d31fb884285ed30e39ec10ef40c864d
Submitter: "Zuul (22348)"
Branch: master

commit d49ce1652d31fb884285ed30e39ec10ef40c864d
Author: Roman Safronov <email address hidden>
Date: Mon Oct 18 09:23:08 2021 +0300

    Fix OVN migration workload creation order

    Currently workload VMs start before subnet is connected to router.
    When DVR is enabled this causes sometimes that one of the VMs is not
    able to get metadata.

    Closes bug: #1947547

    Change-Id: Ifd686d7ff452abd1226fbbc97f499e05102e4596

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/815618

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/815619

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/815620

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/815621

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/815618
Committed: https://opendev.org/openstack/neutron/commit/a805f175aef2fb190c2c2362c360a81458e7094c
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit a805f175aef2fb190c2c2362c360a81458e7094c
Author: Roman Safronov <email address hidden>
Date: Mon Oct 18 09:23:08 2021 +0300

    Fix OVN migration workload creation order

    Currently workload VMs start before subnet is connected to router.
    When DVR is enabled this causes sometimes that one of the VMs is not
    able to get metadata.

    Closes bug: #1947547

    Change-Id: Ifd686d7ff452abd1226fbbc97f499e05102e4596
    (cherry picked from commit d49ce1652d31fb884285ed30e39ec10ef40c864d)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/815621
Committed: https://opendev.org/openstack/neutron/commit/d86a3cdfe77a4e7ca64b57f59000cf0145c7a01a
Submitter: "Zuul (22348)"
Branch: stable/xena

commit d86a3cdfe77a4e7ca64b57f59000cf0145c7a01a
Author: Roman Safronov <email address hidden>
Date: Mon Oct 18 09:23:08 2021 +0300

    Fix OVN migration workload creation order

    Currently workload VMs start before subnet is connected to router.
    When DVR is enabled this causes sometimes that one of the VMs is not
    able to get metadata.

    Closes bug: #1947547

    Change-Id: Ifd686d7ff452abd1226fbbc97f499e05102e4596
    (cherry picked from commit d49ce1652d31fb884285ed30e39ec10ef40c864d)

tags: added: in-stable-xena
tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/815620
Committed: https://opendev.org/openstack/neutron/commit/fbf8cb9347af99803c3ca6019ef21e88b2ad21bc
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit fbf8cb9347af99803c3ca6019ef21e88b2ad21bc
Author: Roman Safronov <email address hidden>
Date: Mon Oct 18 09:23:08 2021 +0300

    Fix OVN migration workload creation order

    Currently workload VMs start before subnet is connected to router.
    When DVR is enabled this causes sometimes that one of the VMs is not
    able to get metadata.

    Closes bug: #1947547

    Change-Id: Ifd686d7ff452abd1226fbbc97f499e05102e4596
    (cherry picked from commit d49ce1652d31fb884285ed30e39ec10ef40c864d)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/815619
Committed: https://opendev.org/openstack/neutron/commit/5f5f2e073a20cee0b9db22cdd82e580d5c265f14
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 5f5f2e073a20cee0b9db22cdd82e580d5c265f14
Author: Roman Safronov <email address hidden>
Date: Mon Oct 18 09:23:08 2021 +0300

    Fix OVN migration workload creation order

    Currently workload VMs start before subnet is connected to router.
    When DVR is enabled this causes sometimes that one of the VMs is not
    able to get metadata.

    Closes bug: #1947547

    Change-Id: Ifd686d7ff452abd1226fbbc97f499e05102e4596
    (cherry picked from commit d49ce1652d31fb884285ed30e39ec10ef40c864d)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 16.4.2

This issue was fixed in the openstack/neutron 16.4.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.1.0

This issue was fixed in the openstack/neutron 19.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.3.0

This issue was fixed in the openstack/neutron 17.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.2.0

This issue was fixed in the openstack/neutron 18.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.0.0.0rc1

This issue was fixed in the openstack/neutron 20.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn train-eol

This issue was fixed in the openstack/networking-ovn train-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.