[RFE] subnet pool can not delete prefixes

[Wenran Xiao]

env: Rocky devstack

I create a subnetpool like this:
[stack@qs11 ~]$ openstack subnet pool show demo-subnetpool4
+-------------------+-------------------------------------------------------------------+
| Field | Value |
+-------------------+-------------------------------------------------------------------+
| address_scope_id | None |
| created_at | 2018-09-17T09:07:04Z |
| default_prefixlen | 26 |
| default_quota | None |
| description | |
| id | ee80a076-ab3a-464d-88bf-a3181c0cf6d8 |
| ip_version | 4 |
| is_default | False |
| max_prefixlen | 32 |
| min_prefixlen | 8 |
| name | demo-subnetpool4 |
| prefixes | 192.168.11.0/24, 198.160.10.0/23, 198.51.100.0/24, 203.0.113.0/24 |
| project_id | 3cc022c8972345aaafc1346343eb4747 |
| revision_number | 6 |
| shared | True |
| tags | |
| updated_at | 2018-09-17T09:52:32Z |
+-------------------+-------------------------------------------------------------------+

The subnet pool have prefixes:
 192.168.11.0/24
 198.160.10.0/23
 198.51.100.0/24
 203.0.113.0/24
The prefixes 198.160.10.0/23 is create by mistake, I try to delete 198.160.10.0/23 prefixes, but I can't find out a method.
usage: neutron subnetpool-update [-h] [--description DESCRIPTION]
                                 [--min-prefixlen MIN_PREFIXLEN]
                                 [--max-prefixlen MAX_PREFIXLEN]
                                 [--default-prefixlen DEFAULT_PREFIXLEN]
                                 [--pool-prefix PREFIXES]
                                 [--is-default {True,False}] [--name NAME]
                                 [--address-scope ADDRSCOPE | --no-address-scope]
                                 SUBNETPOOL

usage: openstack subnet pool set [-h] [--name <name>]
                                 [--pool-prefix <pool-prefix>]
                                 [--default-prefix-length <default-prefix-length>]
                                 [--min-prefix-length <min-prefix-length>]
                                 [--max-prefix-length <max-prefix-length>]
                                 [--address-scope <address-scope> | --no-address-scope]
                                 [--default | --no-default]
                                 [--description <description>]
                                 [--default-quota <num-ip-addresses>]
                                 [--tag <tag>] [--no-tag]
                                 <subnet-pool>

usage: openstack subnet pool unset [-h] [--pool-prefix <pool-prefix>]
                                   [--tag <tag> | --all-tag]
                                   <subnet-pool>
Only have "load prefixes" method.

[Bernard Cafarelli (bcafarel)]

Indeed subnet pools do not support deleting prefixes, the only possible operation is to update them (and only if existing prefixes are a subset of the new prefixes)

[Wenran Xiao (wenran)]

The actual situation is as you described， but If I create a wrong prefixes by mistake, what should I do?
Why does subnet pool not support deleting prefiexs？ Is this some reasons?

[Brian Haley (brian-haley)]

Fix proposed:

https://review.openstack.org/#/c/599633/

[Ryan Tidwell (ryan-tidwell)]

This was a while ago, so my memory is a little fuzzy. I recall there being a bit of contention about the semantics of a PUT on the API to update the prefix list, specifically around how to express removal of a prefix without forcing all PUT's to send the entire list of prefixes. This was not implemented well from the start. In retrospect I would implement a specific subresource (add_prefix or something of the sort) to operate against for prefix updates. Maybe adding this resource and a collorary remove_prefix resource to the API extension would have the benefit of both enabling prefix removal AND a more explicit API. It may be infeasible, but that would leave open the possibility of deprecating PUT support on the prefix_list, making it a read-only attribute. Thoughts?

[Miguel Lavalle (minsel)]

Thanks Ryan for your comment. Here's the patchset that implemented the functionality: https://review.openstack.org/#/c/148698

[Miguel Lavalle (minsel)]

Based on comment #4 above, this is not a bug. This is how the functionality was implemented. I'll tag it as a RFE and, if the submitter decides so, we can continue processing this in the RFE process

[Miguel Lavalle (minsel)]

@Wenran Xiaao,

"... If I create a wrong prefixes by mistake, what should I do?". You can delete the subnet pool and create it again. Why is that not possible solution?

[Wenran Xiao (wenran)]

@Miguel Lavalle

when I delete a subnet pool which already be allocated addresses by subnet, the RESP BODY: {"NeutronError": {"message": "Unable to delete subnet pool: Subnet pool has existing allocations.", "type": "SubnetPoolDeleteError", "detail": ""}}
I need delete subnet first and fall into recursion.
So I don't think it is a good way.

[Miguel Lavalle (minsel)]

@Wenran Xiaao,

Yes, but at that point there is a very good chance that you have allocations from the undesired prefix, so you still have the same problem. Don't you?

[Ryan Tidwell (ryan-tidwell)]

This is actually all working as designed, which is the unfortunate part of this! I think pursuing an RFE is worthwhile, and I would be happy to help with the effort. There are some things we can clean up here. It sounds like there are 2 discrete use cases here, both of which can be pursued independently:

1. Supporting the removal of prefixes from the prefix list
2. Subnet "off-board" (the opposite of subnet onboard) ie taking a subnet that has been allocated from the pool and removing it from the pool. This involves A) removing the prefix allocations for it B) clearing the subnetpool_id attribute of the subnet C) Emitting any address scope change notifications that may be applicable.

[Miguel Lavalle (minsel)]

@Ryan,

Thanks for your comments. So what you are saying is that a user would follow this flow:

1) "off-board" the subnets that were allocated from the the prefix to be deleted
2) Delete the prefix from the list. The delete operation should raise an exception when there are existing subnets on that prefix.

Based on this, I think that 1 could be a follow up to the current effort of subnet on-boarding (https://blueprints.launchpad.net/neutron/+spec/subnet-onboard) and we can pursue 2 in this RFE

[Miguel Lavalle (minsel)]

Let's discuss with the drivers team

[Miguel Lavalle (minsel)]

Drivers team discussed and approved feature 2 as described in note #11 in this RFE on October 5th

[Wenran Xiao (wenran)]

Agree with ”allow to delete a prefix if it is not consumed“. That is just what I want.

[Ryan Tidwell (ryan-tidwell)]

I would suggest taking a look at the implementation of subnet onboard as an example of what the API should look like. In fact, I would like to see prefix additions handled with a similar API. What exists today is not very friendly, it processes updates to the prefix list on simple PUT to the subnetpools resource. It's very difficult to parse intent from that (ie should we remove or add a prefix?). I would like to see an add_prefix and remove_prefix in the ACTION_MAP similar to how we have onboard_network_subnets for subnet onboard.

[1] https://github.com/openstack/neutron-lib/blob/master/neutron_lib/api/definitions/subnet_onboard.py#L42
[2] https://review.openstack.org/#/c/348080/

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/635907

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/635907
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=a6438f427916b01957ff09bc11e959f1ae23ab50
Submitter: Zuul
Branch: master

commit a6438f427916b01957ff09bc11e959f1ae23ab50
Author: Ryan Tidwell <email address hidden>
Date: Fri Feb 8 11:59:37 2019 -0600

    Introduce subnet pool prefix operations extension

    This introduces a new extension that provides API's that enable
    operations against the prefix list of a subnet pool to be more
    explicit. In its current form, the subnetpool extension does not
    clearly define how operations on the prefix list of a subnet pool
    are to be performed and leaves unnecessary ambiguity to be handled
    by implementers of the extension. This extension makes prefix
    management for subnet pools explicit operations, while making support
    for this discoverable for clients.

    Change-Id: Ia84d767f582f28a2fc77799a58f6729a7fd42f34
    Related-Bug: #1792901

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/648197

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/662253

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/648197
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7eb74d2c4aa9bf1db0f165bee64fdf856fc94a2a
Submitter: Zuul
Branch: master

commit 7eb74d2c4aa9bf1db0f165bee64fdf856fc94a2a
Author: Ryan Tidwell <email address hidden>
Date: Fri May 10 13:11:59 2019 -0500

    Implement subnetpool prefix operations

    This changes provides the implementation of the subnetpool prefix
    operations extension. This exposes explicit API's for adding to and
    removing from the prefix list of a subnetpool. Prefixes added to a
    subnetpool are subject to the prefix uniqueness constraints imposed
    by address scopes. Prefixes to be removed from a subnetpool must not
    be allocated to an existing subnet, and the subnet using the prefix
    must be deleted before the prefix can be removed from the subnetpool.

    Change-Id: I76783a4edaf46e184b4dea1d572b89e594bad0ac
    Related-Bug: #1792901

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/662253
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=b7238bd91b6ca17b612b0fe05e12c4fff05b3d33
Submitter: Zuul
Branch: master

commit b7238bd91b6ca17b612b0fe05e12c4fff05b3d33
Author: Ryan Tidwell <email address hidden>
Date: Thu May 30 10:35:09 2019 -0500

    Add API tests for subnet pool prefix operations

    This introduces API tests for the subnetpool_prefix_ops
    extension. These tests assert proper addition and removal of
    prefixes against subnet pools under the various scenarios that
    are supported.

    Related-Bug: #1792901
    Depends-On: https://review.opendev.org/#/c/648197/
    Change-Id: I51564669fc1113556b0927296fa9dd2a8806bce8

[Brian Haley (brian-haley)]

Looks like all the changes for this have merged, marking fixed, please re-open if I missed something.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/906816

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/906816
Committed: https://opendev.org/openstack/neutron/commit/1600d5417507a8f25cd92893adbf76e71aa771e9
Submitter: "Zuul (22348)"
Branch: master

commit 1600d5417507a8f25cd92893adbf76e71aa771e9
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Jan 21 01:00:04 2024 +0000

    [OVN] Add ``subnetpool-prefix-ops`` extension to ML2/OVN mech driver

    Related-Bug: #1792901
    Change-Id: I6b2e86cd252c345b25583651152009baa3320f19

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: unmaintained/yoga
Review: https://review.opendev.org/c/openstack/neutron/+/908818

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.2
Review: https://review.opendev.org/c/openstack/neutron/+/908819

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/2023.1
Review: https://review.opendev.org/c/openstack/neutron/+/908820

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/zed
Review: https://review.opendev.org/c/openstack/neutron/+/908921

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/908819
Committed: https://opendev.org/openstack/neutron/commit/00a9525370bd6e2329af32d5f329b9b8faa3c026
Submitter: "Zuul (22348)"
Branch: stable/2023.2

commit 00a9525370bd6e2329af32d5f329b9b8faa3c026
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Jan 21 01:00:04 2024 +0000

    [OVN] Add ``subnetpool-prefix-ops`` extension to ML2/OVN mech driver

    Related-Bug: #1792901
    Change-Id: I6b2e86cd252c345b25583651152009baa3320f19
    (cherry picked from commit 1600d5417507a8f25cd92893adbf76e71aa771e9)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/908820
Committed: https://opendev.org/openstack/neutron/commit/c019a42086ab1bebc24f87bc131f9faae7ce5cdc
Submitter: "Zuul (22348)"
Branch: stable/2023.1

commit c019a42086ab1bebc24f87bc131f9faae7ce5cdc
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Jan 21 01:00:04 2024 +0000

    [OVN] Add ``subnetpool-prefix-ops`` extension to ML2/OVN mech driver

    Related-Bug: #1792901
    Change-Id: I6b2e86cd252c345b25583651152009baa3320f19
    (cherry picked from commit 1600d5417507a8f25cd92893adbf76e71aa771e9)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/908921
Committed: https://opendev.org/openstack/neutron/commit/b9e3818b6e8905f5717d9888014f7e42141aacf0
Submitter: "Zuul (22348)"
Branch: stable/zed

commit b9e3818b6e8905f5717d9888014f7e42141aacf0
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Jan 21 01:00:04 2024 +0000

    [OVN] Add ``subnetpool-prefix-ops`` extension to ML2/OVN mech driver

    Related-Bug: #1792901
    Change-Id: I6b2e86cd252c345b25583651152009baa3320f19
    (cherry picked from commit 1600d5417507a8f25cd92893adbf76e71aa771e9)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron/+/908818
Committed: https://opendev.org/openstack/neutron/commit/b0e6b059d8de7bad8693826450e7232ef52ff827
Submitter: "Zuul (22348)"
Branch: unmaintained/yoga

commit b0e6b059d8de7bad8693826450e7232ef52ff827
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Sun Jan 21 01:00:04 2024 +0000

    [OVN] Add ``subnetpool-prefix-ops`` extension to ML2/OVN mech driver

    Related-Bug: #1792901
    Change-Id: I6b2e86cd252c345b25583651152009baa3320f19
    (cherry picked from commit 1600d5417507a8f25cd92893adbf76e71aa771e9)