neutron

Load Balancer remains in PENDING_CREATE

Bug #1974052 reported by Fernando Royo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Fernando Royo

Bug Description

While running a heavy load test, where some hundreds of LBs are created in a short period of time, some of them remains in PENDING_CREATE without any apparent error.

[stack@undercloud ~]$ openstack loadbalancer list |grep -i pending
| 073b60fa-e41d-4df7-905e-9455b5fbd39a | 767b70f3-f5fa-496e-ac28-0819ce9411a5-207/cluster-density-207-1 | 172.30.58.222 | PENDING_CREATE | ovn |
| b129391a-ad69-4c0a-b1a3-d8319522cc18 | 767b70f3-f5fa-496e-ac28-0819ce9411a5-157/cluster-density-157-1 | 172.30.161.89 | PENDING_CREATE | ovn |
| 01ac61d0-5c0a-4504-ae2e-07ecfd57a8f9 | 767b70f3-f5fa-496e-ac28-0819ce9411a5-210/cluster-density-210-3 | 172.30.218.204 | PENDING_CREATE | ovn |
| 6244e2c7-2abf-4fc1-a088-4a4145d39177 | 767b70f3-f5fa-496e-ac28-0819ce9411a5-185/cluster-density-185-4 | 172.30.141.108 | PENDING_CREATE | ovn |

See original description
Fernando Royo (froyoredhat)
Changed in neutron:
assignee: nobody → Fernando Royo (froyoredhat)
status: New → In Progress
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/842382

Slawek Kaplonski (slaweq)
Changed in neutron:
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (master)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/842382
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/28d378868fc6ad005f82ef96e573766de68e4d37
Submitter: "Zuul (22348)"
Branch: master

commit 28d378868fc6ad005f82ef96e573766de68e4d37
Author: Fernando Royo <email address hidden>
Date: Wed May 18 17:28:43 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (stable/yoga)

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844055

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (stable/xena)

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844056

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844057

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844058

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844059

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (stable/xena)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844056
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/cbef0f84690f6983ae8457a488dc38465927c2ad
Submitter: "Zuul (22348)"
Branch: stable/xena

commit cbef0f84690f6983ae8457a488dc38465927c2ad
Author: Fernando Royo <email address hidden>
Date: Wed May 18 17:28:43 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19
    (cherry picked from commit 28d378868fc6ad005f82ef96e573766de68e4d37)

tags: added: in-stable-xena
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844057
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/1aa4be5bd11c231a5fac5bb14d7c9347ae504c4a
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 1aa4be5bd11c231a5fac5bb14d7c9347ae504c4a
Author: Fernando Royo <email address hidden>
Date: Wed May 18 17:28:43 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19
    (cherry picked from commit 28d378868fc6ad005f82ef96e573766de68e4d37)

tags: added: in-stable-wallaby
tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844058
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/abc1a24475c7d583f87f7b0035c1af40a3247d51
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit abc1a24475c7d583f87f7b0035c1af40a3247d51
Author: Fernando Royo <email address hidden>
Date: Wed May 18 17:28:43 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19
    (cherry picked from commit 28d378868fc6ad005f82ef96e573766de68e4d37)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844059
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/1b162af1adfd18b14d16d941543e166084d8c185
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 1b162af1adfd18b14d16d941543e166084d8c185
Author: Fernando Royo <email address hidden>
Date: Tue May 31 15:44:04 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052
    (manually cherry picked from commit 28d378868fc6ad005f82ef96e573766de68e4d37)

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (stable/yoga)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/844055
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/4d86a7151c24b5be3c5b6e149b517f3a24c83814
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 4d86a7151c24b5be3c5b6e149b517f3a24c83814
Author: Fernando Royo <email address hidden>
Date: Wed May 18 17:28:43 2022 +0200

    Fix Load balancer remains on PENDING_CREATE

    On LB creation process, when heavy load env situation
    arises, neutron client could return several errors
    that is currently not under control. At this way
    Octavia API is not receiving an update status and LB
    keeps on PENDING_CREATE without any apparent error.

    This patch cover error on request over neutron client
    (subnet, list ports...) in order to update to Octavia
    API with ERROR status in case this situation achieved.

    Closes-Bug: #1974052

    Change-Id: I04edbbd631f63b79c10916ce2391bb0cb95c1c19
    (cherry picked from commit 28d378868fc6ad005f82ef96e573766de68e4d37)

tags: added: in-stable-yoga
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider 1.0.1

This issue was fixed in the openstack/ovn-octavia-provider 1.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider 3.0.0.0rc1

This issue was fixed in the openstack/ovn-octavia-provider 3.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-ovn train-eol

This issue was fixed in the openstack/networking-ovn train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider 1.2.0

This issue was fixed in the openstack/ovn-octavia-provider 1.2.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider 2.1.0

This issue was fixed in the openstack/ovn-octavia-provider 2.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider ussuri-eol

This issue was fixed in the openstack/ovn-octavia-provider ussuri-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider victoria-eom

This issue was fixed in the openstack/ovn-octavia-provider victoria-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.