Creating ports in bulks can be very slow due to many IPAM module conflicts

[Slawek Kaplonski]

When ports are created in bulk, ML2 plugin tries to do everything for all ports in one single DB transaction. That means, that if IP allocation for one of the IPs will fail, whole create_port_bulk method will be retried so it will retry to do everything for all ports.
That may be very inefficient in some use cases, where many ports in bulk are created in same network.

I did simple test:
- create 2 networks with /24 subnet in each network,
- run 24 API requests to create 10 ports in bulk in each request - so in total 240 IPs from subnet will be allocated. All those requests were done in parallel.
- wait how long it will take to have all ports created.

I run my reproducer script 10 times as ipam module is working in some kind of random way so results may be different in various runs.
Results of that simple test are below:

Run Execution time
1 00:01:37
2 00:02:06
3 00:02:08
4 00:02:14
5 00:01:58
6 00:02:37
7 00:01:59
8 00:02:01
9 00:02:39
10 00:01:59
AVG 00:02:07

The execution time here is in fact time of the execution of the longest API request (all of them started together). So there was many retries done internally in Neutron to allocate IP addresses for those ports and client had to wait sometimes more than 150 seconds to get reply for request.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/821727

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/821727
Committed: https://opendev.org/openstack/neutron/commit/82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992
Submitter: "Zuul (22348)"
Branch: master

commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/822794

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/822795

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/822796

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/822797

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/822837

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/822795
Committed: https://opendev.org/openstack/neutron/commit/5e30cb5e24023083eb43c2188ed8da0cad45f151
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 5e30cb5e24023083eb43c2188ed8da0cad45f151
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27
    (cherry picked from commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/822796
Committed: https://opendev.org/openstack/neutron/commit/f830dd3f335ab8a6521b229e437e289e898f1187
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit f830dd3f335ab8a6521b229e437e289e898f1187
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27
    (cherry picked from commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/822794
Committed: https://opendev.org/openstack/neutron/commit/56d2bda51d9750f3aee8d0817b7993b555010571
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 56d2bda51d9750f3aee8d0817b7993b555010571
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27
    (cherry picked from commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/822797
Committed: https://opendev.org/openstack/neutron/commit/5595239435f799926eb3e0397b3bc9d79b62e473
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 5595239435f799926eb3e0397b3bc9d79b62e473
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27
    (cherry picked from commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 17.3.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 18.2.0 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/822837
Committed: https://opendev.org/openstack/neutron/commit/8ca6771d6174e64908c76efe5f12f059b1e97734
Submitter: "Zuul (22348)"
Branch: stable/train

commit 8ca6771d6174e64908c76efe5f12f059b1e97734
Author: Slawek Kaplonski <email address hidden>
Date: Tue Dec 14 12:44:31 2021 +0100

    Allocate IPs in bulk requests in separate transactions

    In the ML2 plugin in create_port_bulk method, we are iterating over
    list of the ports to be created and do everything for all ports in
    single DB transaction (which makes totally sense as this is bulk
    request).
    But one of the things which was done during that huge transaction was
    allocation of the IP addresses for all ports. That action is prone for
    race conditions and can fail often, especially when there is no many IP
    addresses available in the subnet(s) for the ports.
    In case of the error while allocating IP address even for one port from
    the whole bulk request, whole create_port_bulk method was retried so
    allocations (and everything else) for all ports was reverted and started
    from scratch. That takes a lot of time so some requests may be processed
    very long time, like e.g. 2-3 minutes in my tests.

    To reproduce that issue I did simple script which created network with
    /24 subnet and then sent 24 requests to create 10 ports in bulk in each
    request. That was in totall 240 ports created in that subnet.
    I measured time of the creation of all those ports in the current master
    branch (without this patch) and with the patch. Results are like below:

    +-----+---------------+------------+---------------------------+
    | Run | Master branch | This patch | Simulate bulk by creation |
    | | [mm:ss] | [mm:ss] | of 10 ports one by one |
    +-----+---------------+------------+---------------------------+
    | 1 | 01:37 | 01:02 | 00:57 |
    | 2 | 02:06 | 00:40 | 01:03 |
    | 3 | 02:08 | 00:41 | 00:59 |
    | 4 | 02:14 | 00:45 | 00:55 |
    | 5 | 01:58 | 00:45 | 00:57 |
    | 6 | 02:37 | 00:53 | 01:05 |
    | 7 | 01:59 | 00:42 | 00:58 |
    | 8 | 02:01 | 00:41 | 00:57 |
    | 9 | 02:39 | 00:42 | 00:55 |
    | 10 | 01:59 | 00:41 | 00:56 |
    +-----+---------------+------------+---------------------------+
    | AVG | 00:02:07 | 00:00:45 | 00:58 |
    +-----+---------------+------------+---------------------------+

    Conflicts:
        neutron/plugins/ml2/plugin.py

    Closes-Bug: #1954763
    Change-Id: I8877c658446fed155130add6f1c69f2772113c27
    (cherry picked from commit 82aabb0aa962a3c5c5ce5ad1067952d8f3d9f992)
    (cherry picked from commit 5595239435f799926eb3e0397b3bc9d79b62e473)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 20.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/834536

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/834650

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/834651

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/834652

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/834653

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/834656

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/834657

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834536
Committed: https://opendev.org/openstack/neutron/commit/83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd
Submitter: "Zuul (22348)"
Branch: master

commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834650
Committed: https://opendev.org/openstack/neutron/commit/2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834651
Committed: https://opendev.org/openstack/neutron/commit/d2b1e3edaafdf5a2fe4b64457a4caa57a194f858
Submitter: "Zuul (22348)"
Branch: stable/xena

commit d2b1e3edaafdf5a2fe4b64457a4caa57a194f858
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)
    (cherry picked from commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834652
Committed: https://opendev.org/openstack/neutron/commit/b053dfe3844aaf024668847aab3498c3af559dc0
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit b053dfe3844aaf024668847aab3498c3af559dc0
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)
    (cherry picked from commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834653
Committed: https://opendev.org/openstack/neutron/commit/884418183a6e518d755f4e4dd6e05db7e7eef0ab
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 884418183a6e518d755f4e4dd6e05db7e7eef0ab
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)
    (cherry picked from commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834656
Committed: https://opendev.org/openstack/neutron/commit/4f03e76aaafbbc871a6f5c3fc4ecca4298635d6f
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit 4f03e76aaafbbc871a6f5c3fc4ecca4298635d6f
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)
    (cherry picked from commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/834657
Committed: https://opendev.org/openstack/neutron/commit/1254f2371864e10dc66f5aa27031088209f74e9e
Submitter: "Zuul (22348)"
Branch: stable/train

commit 1254f2371864e10dc66f5aa27031088209f74e9e
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Mar 16 16:32:31 2022 +0000

    Remove exception ``IpAddressAllocationNotFound``

    This patch removes the ``IpAddressAllocationNotFound`` exception. This
    exception was raised when a IPAM register was called to be deleted
    but not found.

    As reported in the LP bug, this IPAM register deletion can be called
    several times if a port fails during the creation. The IPAM register
    deletion calls the DB deletion but doesn't raise any exception if the
    register does not exist. The code ensures the IPAM register is
    deleted and there is no need to fail if it is not present anymore.

    This patch also removes the exception catch and try in "update_port",
    that was added in [0] as a fix for [1]. That was added because the
    subnet deletion code involved a port update call [2] during the
    IP allocation deletion, if any port was still present in the subnet.
    Since [3], this code is not needed because the subnet deletion does
    not call a port update anymore.

    [0]https://review.opendev.org/c/openstack/neutron/+/373536
    [1]https://bugs.launchpad.net/neutron/+bug/1622616
    [2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
    [3]https://review.opendev.org/c/openstack/neutron/+/713045

    Closes-Bug: #1965807
    Related-Bug: #1954763
    Related-Bug: #1622616

    Conflicts:
        neutron/ipam/exceptions.py

    Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
    (cherry picked from commit 83b6ce9e9ec2f0261fed630e18cbe9cfa363a0dd)
    (cherry picked from commit 2a6a4ab5dacc2e92c24d68774bf286c9e6ae2d94)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 19.2.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/863881

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/863881
Committed: https://opendev.org/openstack/neutron/commit/f7dd7790f5c6e3149af4680ba521089328d1eb0c
Submitter: "Zuul (22348)"
Branch: master

commit f7dd7790f5c6e3149af4680ba521089328d1eb0c
Author: elajkat <email address hidden>
Date: Fri Nov 4 16:51:03 2022 +0100

    Fix bulk create without mac

    Bulk port create without mac address fails as when Neutron calls
    oslo_utils.netutils.get_ipv6_addr_by_EUI64, as the mac field of the port
    is an ATTR_NOT_SPECIFIED Sentinel() object.
    With some reshuffling of the code to fill the mac field this can be
    fixed.

    Closes-Bug: #1995732
    Related-Bug: #1954763

    Change-Id: Id594003681f4755d8fd1af3b98e281c3109420f6

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/867920

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/867921

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/867920
Committed: https://opendev.org/openstack/neutron/commit/c09f7ec2ee1ade69d3716a13963f6a5fd7714d9d
Submitter: "Zuul (22348)"
Branch: stable/zed

commit c09f7ec2ee1ade69d3716a13963f6a5fd7714d9d
Author: elajkat <email address hidden>
Date: Fri Nov 4 16:51:03 2022 +0100

    Fix bulk create without mac

    Bulk port create without mac address fails as when Neutron calls
    oslo_utils.netutils.get_ipv6_addr_by_EUI64, as the mac field of the port
    is an ATTR_NOT_SPECIFIED Sentinel() object.
    With some reshuffling of the code to fill the mac field this can be
    fixed.

    Conflicts:
      neutron/plugins/ml2/plugin.py

    Closes-Bug: #1995732
    Related-Bug: #1954763

    Change-Id: Id594003681f4755d8fd1af3b98e281c3109420f6
    (cherry picked from commit f7dd7790f5c6e3149af4680ba521089328d1eb0c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/867921
Committed: https://opendev.org/openstack/neutron/commit/2ed8fa503759433136e9e4aef5d9c7b5bb0aa3c5
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 2ed8fa503759433136e9e4aef5d9c7b5bb0aa3c5
Author: elajkat <email address hidden>
Date: Fri Nov 4 16:51:03 2022 +0100

    Fix bulk create without mac

    Bulk port create without mac address fails as when Neutron calls
    oslo_utils.netutils.get_ipv6_addr_by_EUI64, as the mac field of the port
    is an ATTR_NOT_SPECIFIED Sentinel() object.
    With some reshuffling of the code to fill the mac field this can be
    fixed.

    Conflicts:
      neutron/plugins/ml2/plugin.py

    Closes-Bug: #1995732
    Related-Bug: #1954763

    Change-Id: Id594003681f4755d8fd1af3b98e281c3109420f6
    (cherry picked from commit f7dd7790f5c6e3149af4680ba521089328d1eb0c)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/868278

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron/+/868280

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/868278
Committed: https://opendev.org/openstack/neutron/commit/ffd74968956792555de796016c09d87e357f2632
Submitter: "Zuul (22348)"
Branch: stable/xena

commit ffd74968956792555de796016c09d87e357f2632
Author: elajkat <email address hidden>
Date: Fri Nov 4 16:51:03 2022 +0100

    Fix bulk create without mac

    Bulk port create without mac address fails as when Neutron calls
    oslo_utils.netutils.get_ipv6_addr_by_EUI64, as the mac field of the port
    is an ATTR_NOT_SPECIFIED Sentinel() object.
    With some reshuffling of the code to fill the mac field this can be
    fixed.

    Conflicts:
      neutron/plugins/ml2/plugin.py

    Closes-Bug: #1995732
    Related-Bug: #1954763

    Change-Id: Id594003681f4755d8fd1af3b98e281c3109420f6
    (cherry picked from commit f7dd7790f5c6e3149af4680ba521089328d1eb0c)
    (cherry picked from commit 2ed8fa503759433136e9e4aef5d9c7b5bb0aa3c5)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/868280
Committed: https://opendev.org/openstack/neutron/commit/2c854cff4d39dbccfe2e7fa1984ac361e63b60b2
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 2c854cff4d39dbccfe2e7fa1984ac361e63b60b2
Author: elajkat <email address hidden>
Date: Fri Nov 4 16:51:03 2022 +0100

    Fix bulk create without mac

    Bulk port create without mac address fails as when Neutron calls
    oslo_utils.netutils.get_ipv6_addr_by_EUI64, as the mac field of the port
    is an ATTR_NOT_SPECIFIED Sentinel() object.
    With some reshuffling of the code to fill the mac field this can be
    fixed.

    Conflicts:
      neutron/plugins/ml2/plugin.py
      neutron_lib.api.converters.convert_to_sanitized_mac_address
      was only introduced in Xena

    Closes-Bug: #1995732
    Related-Bug: #1954763

    Change-Id: Id594003681f4755d8fd1af3b98e281c3109420f6
    (cherry picked from commit f7dd7790f5c6e3149af4680ba521089328d1eb0c)
    (cherry picked from commit 2ed8fa503759433136e9e4aef5d9c7b5bb0aa3c5)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron train-eol release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron ussuri-eol release.