[OVN] Neutron requests answered with RouterNotFound

Bug #1961967 reported by Elvira García Ruiz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Undecided
Elvira García Ruiz

Bug Description

A recently merged patch [0] has introduced a race condition when creating or updating an OVN Logical Router Port. If the mtu is changed on an external network, the set_gateway_mtu function will look to update all related logical router ports.

There is a small chance in this operation that one of those routers got deleted along the way. This raises a RouterNotFound error and fails.

More info and testing details available on this Bugzilla [1]

[0] https://review.opendev.org/c/openstack/neutron/+/818443
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2055171

Fix proposed: https://review.opendev.org/c/openstack/neutron/+/830509

Changed in neutron:
assignee: nobody → Elvira García Ruiz (elviragr)
Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830509
Committed: https://opendev.org/openstack/neutron/commit/b2421b01e5bc1269458b421643c13b620cdaccaf
Submitter: "Zuul (22348)"
Branch: master

commit b2421b01e5bc1269458b421643c13b620cdaccaf
Author: Elvira García <email address hidden>
Date: Tue Feb 22 18:56:55 2022 +0100

    [OVN] Handle RouterNotFound exception in set_gateway_mtu

    When handling multiple router update/delete petitions, if they are
    executed at the same time, there is a chance that the set_gateway_mtu
    tries to update a no-longer-existing router, triggering a
    RouterNotFound error from the _get_router_ports function. Now, if there
    is not a router, the _get_router_ports function will return an empty
    list instead of raising the error.

    Closes-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: Ic54214ca4965274cd58c1d3a278d8f3ffcda0a64

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/830666

tags: added: ovn
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/830864

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/830865

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/830866

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/830867

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830865
Committed: https://opendev.org/openstack/neutron/commit/86c153d57fa2579be68b59ae9b29f89ed0c347f2
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 86c153d57fa2579be68b59ae9b29f89ed0c347f2
Author: Elvira García <email address hidden>
Date: Tue Feb 22 18:56:55 2022 +0100

    [OVN] Handle RouterNotFound exception in set_gateway_mtu

    When handling multiple router update/delete petitions, if they are
    executed at the same time, there is a chance that the set_gateway_mtu
    tries to update a no-longer-existing router, triggering a
    RouterNotFound error from the _get_router_ports function. Now, if there
    is not a router, the _get_router_ports function will return an empty
    list instead of raising the error.

    Closes-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: Ic54214ca4965274cd58c1d3a278d8f3ffcda0a64
    (cherry picked from commit b2421b01e5bc1269458b421643c13b620cdaccaf)

tags: added: in-stable-wallaby
tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830864
Committed: https://opendev.org/openstack/neutron/commit/c0c292efbc5a4f87041a72fbe82f3239507b1342
Submitter: "Zuul (22348)"
Branch: stable/xena

commit c0c292efbc5a4f87041a72fbe82f3239507b1342
Author: Elvira García <email address hidden>
Date: Tue Feb 22 18:56:55 2022 +0100

    [OVN] Handle RouterNotFound exception in set_gateway_mtu

    When handling multiple router update/delete petitions, if they are
    executed at the same time, there is a chance that the set_gateway_mtu
    tries to update a no-longer-existing router, triggering a
    RouterNotFound error from the _get_router_ports function. Now, if there
    is not a router, the _get_router_ports function will return an empty
    list instead of raising the error.

    Closes-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: Ic54214ca4965274cd58c1d3a278d8f3ffcda0a64
    (cherry picked from commit b2421b01e5bc1269458b421643c13b620cdaccaf)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830866
Committed: https://opendev.org/openstack/neutron/commit/69182db50e841f159e7b0be5bcabccdb76173120
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 69182db50e841f159e7b0be5bcabccdb76173120
Author: Elvira García <email address hidden>
Date: Tue Feb 22 18:56:55 2022 +0100

    [OVN] Handle RouterNotFound exception in set_gateway_mtu

    When handling multiple router update/delete petitions, if they are
    executed at the same time, there is a chance that the set_gateway_mtu
    tries to update a no-longer-existing router, triggering a
    RouterNotFound error from the _get_router_ports function. Now, if there
    is not a router, the _get_router_ports function will return an empty
    list instead of raising the error.

    OVNL3RouterPlugin has no attribute '_nb_ovn'
    FakeOvsdbNbOvnIdl has no attribute 'lrp_set_options'
    Conflicts:
        neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py

    Closes-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: Ic54214ca4965274cd58c1d3a278d8f3ffcda0a64
    (cherry picked from commit b2421b01e5bc1269458b421643c13b620cdaccaf)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830867
Committed: https://opendev.org/openstack/neutron/commit/4e8ee06f570f5b9158e9ffefe0a67fb1e668761c
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 4e8ee06f570f5b9158e9ffefe0a67fb1e668761c
Author: Elvira García <email address hidden>
Date: Tue Feb 22 18:56:55 2022 +0100

    [OVN] Handle RouterNotFound exception in set_gateway_mtu

    When handling multiple router update/delete petitions, if they are
    executed at the same time, there is a chance that the set_gateway_mtu
    tries to update a no-longer-existing router, triggering a
    RouterNotFound error from the _get_router_ports function. Now, if there
    is not a router, the _get_router_ports function will return an empty
    list instead of raising the error.

    OVNL3RouterPlugin has no attribute '_nb_ovn'
    FakeOvsdbNbOvnIdl has no attribute 'lrp_set_options'
    Conflicts:
        neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py

    Closes-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: Ic54214ca4965274cd58c1d3a278d8f3ffcda0a64
    (cherry picked from commit b2421b01e5bc1269458b421643c13b620cdaccaf)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/830666
Committed: https://opendev.org/openstack/neutron/commit/a51aff743c4659347f39efa1fd36ed883a37766f
Submitter: "Zuul (22348)"
Branch: master

commit a51aff743c4659347f39efa1fd36ed883a37766f
Author: Elvira García <email address hidden>
Date: Wed Feb 23 17:54:36 2022 +0100

    [OVN][Cosmetic] Improve gen_router_port_options

    The gen_router_port_options now has a try/catch operation that could be
    better formulated as jlibosva stated in a comment in
    https://review.opendev.org/c/openstack/neutron/+/830509

    Related-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I679bbb5aa59d0a71d14bd5540d33eada53054b18

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/xena)

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/831571

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/wallaby)

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/831572

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/victoria)

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/831573

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/831571
Committed: https://opendev.org/openstack/neutron/commit/bd738eadd15f007dcc3a3c8bbb514335517ba1ff
Submitter: "Zuul (22348)"
Branch: stable/xena

commit bd738eadd15f007dcc3a3c8bbb514335517ba1ff
Author: Elvira García <email address hidden>
Date: Wed Feb 23 17:54:36 2022 +0100

    [OVN][Cosmetic] Improve gen_router_port_options

    The gen_router_port_options now has a try/catch operation that could be
    better formulated as jlibosva stated in a comment in
    https://review.opendev.org/c/openstack/neutron/+/830509

    Related-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I679bbb5aa59d0a71d14bd5540d33eada53054b18
    (cherry picked from commit a51aff743c4659347f39efa1fd36ed883a37766f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/831572
Committed: https://opendev.org/openstack/neutron/commit/e6b7689003e64a80a9e7df5e401f72238b4b4dd3
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit e6b7689003e64a80a9e7df5e401f72238b4b4dd3
Author: Elvira García <email address hidden>
Date: Wed Feb 23 17:54:36 2022 +0100

    [OVN][Cosmetic] Improve gen_router_port_options

    The gen_router_port_options now has a try/catch operation that could be
    better formulated as jlibosva stated in a comment in
    https://review.opendev.org/c/openstack/neutron/+/830509

    Related-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I679bbb5aa59d0a71d14bd5540d33eada53054b18
    (cherry picked from commit a51aff743c4659347f39efa1fd36ed883a37766f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/831573
Committed: https://opendev.org/openstack/neutron/commit/2b8edb859328b785217f6ff3fc662e7547ff07f2
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 2b8edb859328b785217f6ff3fc662e7547ff07f2
Author: Elvira García <email address hidden>
Date: Wed Feb 23 17:54:36 2022 +0100

    [OVN][Cosmetic] Improve gen_router_port_options

    The gen_router_port_options now has a try/catch operation that could be
    better formulated as jlibosva stated in a comment in
    https://review.opendev.org/c/openstack/neutron/+/830509

    Related-Bug: #1961967
    Signed-off-by: Elvira García <email address hidden>
    Change-Id: I679bbb5aa59d0a71d14bd5540d33eada53054b18
    (cherry picked from commit a51aff743c4659347f39efa1fd36ed883a37766f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 20.0.0.0rc1

This issue was fixed in the openstack/neutron 20.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 17.4.0

This issue was fixed in the openstack/neutron 17.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.3.0

This issue was fixed in the openstack/neutron 18.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 19.2.0

This issue was fixed in the openstack/neutron 19.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn train-eol

This issue was fixed in the openstack/networking-ovn train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron ussuri-eol

This issue was fixed in the openstack/neutron ussuri-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.