Loadbalacer stuck in status PENDING_DELETE if TLS storage unavailable in cascade deletion

Bug #2077348 reported by Evgeniy Bykov
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Status tracked in Epoxy
Antelope
New
Undecided
Unassigned
Bobcat
New
Undecided
Unassigned
Caracal
New
Undecided
Unassigned
Dalmation
New
Undecided
Unassigned
Epoxy
New
Undecided
Unassigned
Yoga
New
Undecided
Unassigned
octavia
Fix Released
Undecided
Evgeniy Bykov
octavia (Ubuntu)
Status tracked in Plucky
Focal
New
Undecided
Unassigned
Jammy
New
Undecided
Unassigned
Noble
New
Undecided
Unassigned
Oracular
New
Undecided
Unassigned
Plucky
New
Undecided
Unassigned

Bug Description

Loadbalacer stuck in status PENDING_DELETE if TLS cert unavailable

1. Create load balancer with TERMINATED_HTTPS listener
2. Disable your TLS storage, or delete cert from storage
3. Try to delete loadbalancer with cascade flag

Error on logs:

```
Unable to retrieve certificate(s) due to Could not retrieve certificate: <some id>
Exception during message handling
```

```
Traceback (most recent call last):
File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming, res = self.dispatcher.dispatch(message),
File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch, return self._do_dispatch(endpoint, method, ctxt, args),
File "/var/lib/openstack/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch, result = func(ctxt, **new_args),
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/queue/v2/endpoints.py", line 56, in delete_load_balancer, self.worker.delete_load_balancer(loadbalancer, cascade),
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/controller_worker.py", line 387, in delete_load_balancer, listeners = flow_utils.get_listeners_on_lb(db_lb),
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/controller/worker/v2/flows/flow_utils.py", line 52, in get_listeners_on_lb, prov_listener = provider_utils.db_listener_to_provider_listener(),
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 182, in db_listener_to_provider_listener, new_listener_dict = listener_dict_to_provider_dict(),
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 261, in listener_dict_to_provider_dict, with excutils.save_and_reraise_exception() as ctxt:,
File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__, self.force_reraise(),
File "/var/lib/openstack/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise, raise self.value,
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/api/drivers/utils.py", line 258, in listener_dict_to_provider_dict, cert_dict = cert_parser.load_certificates_data(cert_manager)
File "/var/lib/openstack/lib/python3.10/site-packages/octavia/common/tls_utils/cert_parser.py", line 381, in load_certificates_data, raise exceptions.CertificateRetrievalException(, octavia.common.exceptions.CertificateRetrievalException: Could not retrieve certificate: ]

```

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/octavia/+/926564

Changed in octavia:
status: New → In Progress
Changed in octavia:
assignee: nobody → Evgeniy Bykov (lydina-pavuh)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (master)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/926564
Committed: https://opendev.org/openstack/octavia/commit/ec9a50599012b99deaf8bb26683fedcd381db1a6
Submitter: "Zuul (22348)"
Branch: master

commit ec9a50599012b99deaf8bb26683fedcd381db1a6
Author: Evgeniy Bykov <email address hidden>
Date: Mon Aug 19 20:20:38 2024 +0300

    Fix loadbalancer stuck in cascade delete

    - Fix loadbalancer stuck in PENDING_DELETE in cascade delete
    with TERMINATED_HTTPS listener if TLS storage not available

    Closes-Bug: #2077348
    Change-Id: Iae2b075de5412bb183db6a21f7f8376801853e00
    Signed-off-by: Evgeniy Bykov <email address hidden>

Changed in octavia:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/2023.1)

Fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/octavia/+/928322

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/2023.2)

Fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/octavia/+/928323

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to octavia (stable/2024.1)

Fix proposed to branch: stable/2024.1
Review: https://review.opendev.org/c/openstack/octavia/+/928324

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 15.0.0.0rc1

This issue was fixed in the openstack/octavia 15.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/2023.1)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/928322
Committed: https://opendev.org/openstack/octavia/commit/97603dde566364c63e381c9bb557e8a430656dbc
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit 97603dde566364c63e381c9bb557e8a430656dbc
Author: Evgeniy Bykov <email address hidden>
Date: Mon Aug 19 20:20:38 2024 +0300

    Fix loadbalancer stuck in cascade delete

    - Fix loadbalancer stuck in PENDING_DELETE in cascade delete
    with TERMINATED_HTTPS listener if TLS storage not available

    Closes-Bug: #2077348
    Change-Id: Iae2b075de5412bb183db6a21f7f8376801853e00
    Signed-off-by: Evgeniy Bykov <email address hidden>
    (cherry picked from commit ec9a50599012b99deaf8bb26683fedcd381db1a6)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/2024.1)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/928324
Committed: https://opendev.org/openstack/octavia/commit/3af5401b81d7132af7939b64e7ef865629a47aad
Submitter: "Zuul (22348)"
Branch: stable/2024.1

commit 3af5401b81d7132af7939b64e7ef865629a47aad
Author: Evgeniy Bykov <email address hidden>
Date: Mon Aug 19 20:20:38 2024 +0300

    Fix loadbalancer stuck in cascade delete

    - Fix loadbalancer stuck in PENDING_DELETE in cascade delete
    with TERMINATED_HTTPS listener if TLS storage not available

    Closes-Bug: #2077348
    Change-Id: Iae2b075de5412bb183db6a21f7f8376801853e00
    Signed-off-by: Evgeniy Bykov <email address hidden>
    (cherry picked from commit ec9a50599012b99deaf8bb26683fedcd381db1a6)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to octavia (stable/2023.2)

Reviewed: https://review.opendev.org/c/openstack/octavia/+/928323
Committed: https://opendev.org/openstack/octavia/commit/09ea1e991a9acddece7cca0c9d1ef6c6a69be37b
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 09ea1e991a9acddece7cca0c9d1ef6c6a69be37b
Author: Evgeniy Bykov <email address hidden>
Date: Mon Aug 19 20:20:38 2024 +0300

    Fix loadbalancer stuck in cascade delete

    - Fix loadbalancer stuck in PENDING_DELETE in cascade delete
    with TERMINATED_HTTPS listener if TLS storage not available

    Closes-Bug: #2077348
    Change-Id: Iae2b075de5412bb183db6a21f7f8376801853e00
    Signed-off-by: Evgeniy Bykov <email address hidden>
    (cherry picked from commit ec9a50599012b99deaf8bb26683fedcd381db1a6)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/octavia 2023.1-eom

This issue was fixed in the openstack/octavia 2023.1-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.