[RFE] Enable adoption of an existing subnet into a subnetpool

[Ryan Tidwell]

It is currently not possible to take an existing subnet and move it into a subnetpool or address scope. The subnetpool_id attribute on subnets is not updateable, meaning that once a subnet is created it cannot be moved between subnetpools or address scopes. If a subnet was created without a subnetpool_id, it still cannot be moved into a subnetpool or address scope. This is a barrier to the adoption features like subnetpools, address scopes, and BGP. Many users likely have existing subnets that they would prefer to not destroy and re-create just for the purpose of being able to use these features.

This RFE proposes making the subnetpool_id attribute updateable on subnets. To conform to the uniqueness guarantees provided by subnetpools and address scopes, the subnet cidr will be checked across the subnetpool prefix list and across the entire address scope (if applicable) to ensure the cidr has not already been allocated from either the subnetpool or the address scope.

Adoption of a subnet by a subnetpool would have the following behavior:

1. If the subnet cidr has been configured in the address scope as a pool prefix in a different subnetpool than the specified subnetpool, the adoption into the specified subnetpool will be blocked

2. If the subnet cidr has been configured in the specified subnetpool as a pool prefix and the pool prefix has not been allocated, the subnetpool_id of the subnet will be updated and the cidr will be seen by the subnetpool as allocated.

3. If the subnet cidr has been configured in the specified subnetpool as a pool prefix and the pool prefix has been allocated, adoption into the specified subnet will be blocked.

4. If the subnet cidr has not been configured as a pool prefix in the specified subnetpool and the pool prefix is not configured as a pool prefix in any other subnetpools in the address scope, the cidr will be added as a pool prefix to the specified subnetpool and marked as allocated

5. If a subnet allocated from a subnetpool has its subnetpool_id updated, the cidr is released back to the subnetpool upon completion of the update. Moving a subnet between subnetpools is subject to the same restrictions as explained above.

[Cedric Brandily (cbrandily)]

It should allow to abandon the subnet (should be easier as there is no check to perform)

[Ryan Tidwell (ryan-tidwell)]

Orphaning a subnet so that it belongs to no subnetpool should be allowed. In that case we would simply release the prefix back to the pool upon updating subnetpool_id, I didn't explain that very well. Thanks for pointing that out.

[Carl Baldwin (carl-baldwin)]

I discussed this at breakfast one morning with armax in Bristol.

I still think that this could be useful but it cannot be back-ported to Mitaka or earlier. If we provide an offline script, we could provide it for older clouds. An offline script might have trouble with L3 agents not getting updated with the new address scopes.

There is an issue with multiple subnets on the same network. If the change results in the subnet being in a different scope, then all of the subnets on the same network would have to be changed at the same time to avoid a network with different address scopes across subnets. That makes it difficult for me to imagine how a network with multiple subnets could be changed through the API because they'd have to be change atomically.

[Armando Migliaccio (armando-migliaccio)]

I remember the drizzly morning...let's discuss this during the meeting to keep others in the loop

[Armando Migliaccio (armando-migliaccio)]

Status update:

http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-03-31-22.00.log.html#l-285

[Armando Migliaccio (armando-migliaccio)]

In general I think this is a sensible thing to do but the devil is in the details. I think this need a full spec so that we can think carefully of all the steps involved.

[Reedip (reedip-banerjee-deactivatedaccount)]

Just to keep it as a note: Would impact the client side as well ( Keeping OSC migration in mind)

[Brian Stajkowski (brian-stajkowski)]

So, my question here is why approach it from the perspective of creating the subnetpool then going through and updating all subnets with the subnetpool_id. Would it be simpler to perform this update for the user when the subnetpool is created?

We know that the subnets can't already overlap, so why not detect what subnets fall into the subnetpool prefix range and automatically update the subnet's subnetpool_id? These seems like a simpler approach rather than adjust the current subnets resource.

[Brian Stajkowski (brian-stajkowski)]

Although, now with subnetpool being out and in the wild, I can see the case where we would need to allow subnetpool_id to be updated if they created a subnetpool already, rather than delete and recreate it, especially if they already have subnets associated. But, how many operators have painted themselves into a corner like this?

[Ryan Tidwell (ryan-tidwell)]

The problem is that subnets can overlap in neutron when you use the null address scope. This is legacy behavior. It's not as simple as just gathering up subnets with matching CIDR's. You also have to account for address scopes. The reason this is hung up is really that I haven't had time to work on it. I think the approach I've taken will work with some tweaks. We could change to API to allow adoption of network subnets in bulk to make things easier. Unfortunately we have to adopt subnets by network because of the constraint that all subnets of the same address family must belong to the same address scope.

[Brian Stajkowski (brian-stajkowski)]

This makes more sense now. Basically tier the check from the subnetpool all the way up to the address scope and if the prefix hasn't been allocated, add the subnetpool_id. Understood.

[Carl Baldwin (carl-baldwin)]

I'll try to summarize my thoughts here. Most of this has been discussed above but this gets it all in one place.

Neutron enforces the invariant that all of the subnets on a network must come from the same subnetpool. So, the obvious change to the API -- make subnetpool_id updatable on the subnet -- wouldn't really work because the call to update the first subnet would leave all of the other subnets on a different subnetpool this breaking the invariant.

Need an API call to update all of the subnets on a network at once. I had thought about adding an updateable subnetpool_id on the network but something about that didn't seem right. Then, we thought about adding something new. It might look like PUT /networks/{network_id}/adopt_subnets or something like that. Ryan Tidwell had some good thoughts on this.

Anyway, once the API has been decided, then there are the mechanics of adopting new subnets into a subnetpool. Neutron currently checks that new subnets being added to a pool do not overlap with any available subnet block in the address scope that owns the pool. This new API would have to check that the subnets being added do not overlap any allocated subnets from any subnetpool in the address scope also.

If the subnets being added to the pool aren't defined as available subnets in the pool, I think this is ok. I wouldn't fail the operation because of this. In fact, forcing the user to add them first as available in the subnetpool would create another race condition. They just can't overlap any subnets that are allocated. You would just end up with a situation where the set of allocated subnets from the pool is not a subset of the set of available subnets in the pool. This is actually ok but if you'd feel better, you can add the missing ones to the set of available subnets.

To get a little more technical, here are some things that should be checked before adopting the subnets in to the pool. `<>` means sets are disjoint, no overlap. `new_subnets` is the union of subnets in the network to adopt. `allocated_in_scope` is the union of subnets currently allocated across all pools in the scope. `available_in_scope` is the union of all subnets defined as available in all of the pools in the scope. `available_in_pool` is the union of all subnets defined as available in the pool we're adopting into.

  new_subnets <> allocated_in_scope
  new_subnets <> (available_in_scope - available_in_pool)

[Poonam Ghosh (poonam-ghosh)]

We are planning to pick this Wishlist Bug for our analysis and contribution.
This is to intimate our participation on this Bugs analysis and Implementation.

[Reedip (reedip-banerjee-deactivatedaccount)]

Hi Poonam,
Thanks for your update.
Trevor McCasland and I are also working on this issue ( https://review.openstack.org/#/c/407697/ )
We are awaiting the response from the Drivers team for the future course of action.

[Rob P (rpkr)]

Hi Brian Stajkowski

> But, how many operators have painted themselves into a corner like this?

We are one of these operators, following your progress for an answer to our problems :)

Can we be of some assistance here in anyway?