[ovn-octavia-provider] Octavia LB stuck in PENDING_UPDATE after creation

[Jack Hodgkiss]

Wallaby OpenStack deployment using OpenStack Kayobe

Running on Ubuntu Focal

Relevant package version:

    - octavia-lib: 2.3.1
    - neutron-lib: 2.10.2
    - ovn-octavia-provider: 1.0.2.dev5

I have encountered a bug where after creating an Octavia load balancer it gets stuck and cannot be deleted.

Attempt in Horizon to delete the load balancer are met with the following
Error: Unable to delete Load Balancer: test_ovn_lb. It also reports Provisioning Status: Pending Update.

When attempting to delete via the openstack client I get this response.

(openstackclient-venv) [~] openstack loadbalancer delete 64951486-8143-4a17-a88b-9f576688e662
Validation failure: Cannot delete Load Balancer 64951486-8143-4a17-a88b-9f576688e662 - it has children (HTTP 400) (Request-ID: req-5bf53e03-d33d-4995-88fb-8617060afdf4)

(openstackclient-venv) [~] openstack loadbalancer delete 64951486-8143-4a17-a88b-9f576688e662 --cascade
Invalid state PENDING_UPDATE of loadbalancer resource 64951486-8143-4a17-a88b-9f576688e662 (HTTP 409) (Request-ID: req-cd917d82-67cd-4704-b6d2-032939e08d88)

In the octavia-api.log the following error message is logged in the moments prior to getting stuck in this state. https://paste.opendev.org/show/bkKWy2WkjC9fo05kOFE3/

The only solution to this problem that I have found that works is to edit the Octavia table to change the current Pending State to ERROR.

use octavia
UPDATE load_balancer SET provisioning_status = 'ERROR' WHERE provisioning_status LIKE "PENDING_UPDATE";

This manual edit of the database then allows for the removal of the load balancer via the API:

openstack loadbalancer delete id-here --cascade

This bug is not blocking however it would nice to prevent this from happening again.

[Fernando Royo (froyoredhat)]

Hi Jack,

Could you provide some logs from server (file octavia.log) or give more info about the config of the LB that generates the error? (listener/pool/members, subnets involved...)

Just to figure out the context, because several improvementes had been done during last months in order to avoid this stuck situations.

[Jack Hodgkiss (jackhodgkiss)]

Hello Fernando,

Here are some additional logs from the server at the time the issue occurred. https://paste.opendev.org/show/baLngfpx1jYvcmUTS7Zw/

This occurred as I attempted to create the load balancer via Horizon. Which I now know is not possible as Horizon currently doesn't support Octavia based load balancers.

[Fernando Royo (froyoredhat)]

Hi Jack,

I think the issue is the protocol specified, OVN Octavia provider just support SOURCE_IP_PORT protocol for the pool, it looks like the Horizon UI is sending SOURCE_IP and the error is the expected behaviour, OVN provider is rejecting the request with the msg:

'octavia.common.exceptions.ProviderUnsupportedOptionError: Provider 'ovn' does not support a requested option: OVN provider does not support SOURCE_IP algorithm
'

if you replicate the same command using SOURCE_IP_PORT instead of you will get a success request.

So I think we should close this bug and request support from the Horizon panel, I don't have a devstack vm with Horizon UI deployed right now to test it, but I trust your testing enough to go ahead with that proposal.