LB creation failed due to address already in use

[Fernando Royo]

While running a heavy load test, where create CRUD operations over load balancers and editing all pending items (listener/pool/member). We are getting error "IP_ADDRESS already allocated in subnet". Apparently because the address is being used by a Port for the load-balancer, but the load-balancer doesn't exist in a post error check.

e.g. when create loadbalancer is sent we get this error from Octavia API:

octavia.common.exceptions.ProviderDriverError: Provider 'ovn' reports error: IP address 172.30.125.1 already allocated in subnet beb4de56-a2b1-487b-9ab5-e10a0ad0d7ac

[stack@undercloud ~]$ openstack port list | grep 172.30.125.1
| 6a9c24a2-7907-4b5c-b406-f74ba5a05820 | ovn-lb-vip-22efa8a7-460c-443f-8f53-ac53929e8637 | fa:16:3e:ea:e5:c4 | ip_address='172.30.125.1', subnet_id='beb4de56-a2b1-487b-9ab5-e10a0ad0d7ac' | DOWN |

[stack@undercloud ~]$ openstack loadbalancer list --vip-port-id ovn-lb-vip-22efa8a7-460c-443f-8f53-ac53929e8637
[stack@undercloud ~]$

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/842107

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/842107
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/cd3264a07e047b589e4064855169c37f17343e60
Submitter: "Zuul (22348)"
Branch: master

commit cd3264a07e047b589e4064855169c37f17343e60
Author: Fernando Royo <email address hidden>
Date: Tue May 17 15:32:24 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848578

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848579

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848580

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848582

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848583

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848578
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/3a8aa3349adab85d81614f2d079314de9ecaa9e3
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 3a8aa3349adab85d81614f2d079314de9ecaa9e3
Author: Fernando Royo <email address hidden>
Date: Tue May 17 15:32:24 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d
    (cherry picked from commit cd3264a07e047b589e4064855169c37f17343e60)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848580
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/fb765e856f6f815feaabf4b84c94a6d28eb862fb
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit fb765e856f6f815feaabf4b84c94a6d28eb862fb
Author: Fernando Royo <email address hidden>
Date: Tue May 17 15:32:24 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d
    (cherry picked from commit cd3264a07e047b589e4064855169c37f17343e60)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848582
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/99fa23ffe7bd154a1640ef90c9f28a5432a68bd5
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 99fa23ffe7bd154a1640ef90c9f28a5432a68bd5
Author: Fernando Royo <email address hidden>
Date: Tue May 17 15:32:24 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d
    (cherry picked from commit cd3264a07e047b589e4064855169c37f17343e60)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848579
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/aae410fc85e7ca9341d37961731d16af369f2ab2
Submitter: "Zuul (22348)"
Branch: stable/xena

commit aae410fc85e7ca9341d37961731d16af369f2ab2
Author: Fernando Royo <email address hidden>
Date: Tue May 17 15:32:24 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d
    (cherry picked from commit cd3264a07e047b589e4064855169c37f17343e60)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/848583
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/4d15d0d2f66e0cc5a26ff37527aa62d770a6a5e3
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 4d15d0d2f66e0cc5a26ff37527aa62d770a6a5e3
Author: Fernando Royo <email address hidden>
Date: Mon Jul 4 10:43:54 2022 +0200

    Delete vip port if create_vip_port raise exception

    Sometimes on heavy load over Neutron, operations over
    neutron client could raise a NeutronClientException that
    could cover, e.g. a timeout. In those case, sometimes the
    request finish after the client timeout and we have
    leftover resource.

    create_vip_port is called from Octavia API just before
    the create LB request, if this call raise a timeout
    maybe the vip port is finally created and linked to a LB
    that is not going to be created, so we have this port
    unuse.

    This patch will try to remove the leftover port for those
    situations.

    (manually cherry picked from commit
    cd3264a07e047b589e4064855169c37f17343e60)

    Closes-Bug: #1973765
    Change-Id: Iad9839ce99d50b969924b3fe369301b6dfed369d

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider 1.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider 3.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-ovn train-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider 1.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider 2.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider ussuri-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ovn-octavia-provider victoria-eom release.