Disallow users to allocate gateway ip of external subnets as floating ip

[Sebastian Lohff]

Currently a user can allocate the gateway ip of an external network as a floating ip. This is possible, as the only validation on a user specified ip address is done by the ipam module, which checks that an ip is in the range of the subnet(s) and that it is not already allocated. Because OpenStack has no port for the external gateway the subnet of an external network is marked as free.

This is a problem because now a user can allocate an IP address that might be otherwise in use (externally of OpenStack / inside a provider network). Depending on the network plugins used, the user could either end up with an unusable floating ip or (in the worst case) create something that arps for this IP and redirects traffic away from the original gateway, causing an outage. Therefore I propose we forbid users from allocating floatingips that are also the gateway ip in a floating ip network. Note that OpenStack would not allocate the gateway ip itself, as it only allocates from the subnet's allocation pool by default.

To fix this I'd propose we either explicitly deny using the gateway ip or require the user-specified IP for a subnet to be from the allocation pool. I'd be happy to provide a patch once we have decided how to approach this.

This can be recreated with a simple cli command: openstack floating ip create $fip_network --floating-ip-address $gateway_ip_of_subnet

A similar bug was filed and fixed for putting routers into provider networks: https://bugs.launchpad.net/neutron/+bug/1757482

Breaking testcase (neutron/tests/unit/extensions/test_l3.py):
class L3NatTestCaseBase(L3NatTestCaseMixin):
    def test_create_floatingip_on_external_subnet_gateway_fails(self):
        with self.subnet(cidr='11.0.0.0/24') as public_sub:
            self._set_net_external(public_sub['subnet']['network_id'])
            self._make_floatingip(
                self.fmt,
                public_sub['subnet']['network_id'],
                floating_ip=public_sub['subnet']['gateway_ip'],
                http_status=exc.HTTPBadRequest.code)

Preliminary discussion in IRC:
https://meetings.opendev.org/irclogs/%23openstack-neutron/%23openstack-neutron.2022-02-01.log.html#t2022-02-01T15:02:10

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello Sebastian:

First of all, let me pointing out that [1] was not the same problem as in this bug. In [1], the router port had an IP address outside the subnet CIDR; that was an error. This is not the case here, were the FIP receives an IP address in this range.

IMO, we should allow this. The GW IP is not a Neutron resource (an IP allocation, associated to a port). This is a parameter of the subnet. We could write a warning message in the logs, that's all. The user should be able to assign this IP to a FIP (for whatever use case I can't image now). Of course, that's my own opinion only.

Regards.

[1]https://bugs.launchpad.net/neutron/+bug/1757482

[Brian Haley (brian-haley)]

Random comment - wouldn't you typically remove the gateway IP from the allocation pool during subnet creation? There is cli/api support for that:

--allocation-pool start=<ip-address>,end=<ip-address>

devstack typically does that when it configures the external network/subnet, but in an actual deployment it's useful since there could be a number of IP addresses you do not want to use.

[Johannes Kulik (jkulik)]

> wouldn't you typically remove the gateway IP from the allocation pool during subnet creation?

Yes, it's still possible to explicitly request an IP of a subnet, even if it's not part of the allocation pool. That's where the suggestion to "require the user-specified IP for a subnet to be from the allocation pool" comes from.

[Sebastian Lohff (sebageek)]

> First of all, let me pointing out that [1] was not the same problem as in this bug. In [1],
> the router port had an IP address outside the subnet CIDR; that was an error. This is not
> the case here, were the FIP receives an IP address in this range.

The bug report in [1] sais "IP address for a router interface allowed outside the allocation range of subnet". The IP address was inside the subnet CIDR, but not inside the allocation range and this was deemed to be a problem. In this case Neutron chose the gateway ip for the router, which is inside the subnet's CIDR, but outside of the subnet's allocation pool. The python test I pasted shows similar behavior; if you print the public_sub you have a gateway outside the allocation pool, which is allocatable as a FIP:

> 'gateway_ip': '11.0.0.1', 'cidr': '11.0.0.0/24', 'allocation_pools': [{'start': '11.0.0.2', 'end': '11.0.0.254'}]

So, what Brian describes would solve my problem if this behavior would be implemented in OpenStack. Implementing it would probably mean writing extra validation checks for create_floatingip() with user-specified floating_ip_address.

[Lajos Katona (lajos-katona)]

By implementing in Openstack you mean that there should be a default allocation pool for the subnet which excludes the gateway IP?
The api-ref says so (https://docs.openstack.org/api-ref/network/v2/index.html?expanded=create-subnet-detail#id279 ), not sure if this is exactly what you need:

" If allocation_pools are not specified, OpenStack Networking automatically allocates pools for covering all IP addresses in the CIDR, excluding the address reserved for the subnet gateway by default."

[Sebastian Lohff (sebageek)]

What I want is that the user is unable to allocate the gateway IP as a floating IP.

> " If allocation_pools are not specified, OpenStack Networking automatically allocates pools for covering all IP addresses in the CIDR, excluding the address reserved for the subnet gateway by default."

This describes what I want, but does not work if the user specifies an IP address. The problem is that the user can specify (and therefore allocate) the gateway IP of the CIDR. The gateway IP is excluded in the allocation pool of my subnet, but the user is _still able to allocate IPs outside the allocation pool_. So what I'm proposing is to not allow the user to allocate floating IPs outside of the allocation pool of a subnet (desired state) instead of allowing them to allocate everything in the CIDR including the gateway IP even if outside of the allocation pool (current state).

The test attached in the original post should illustrate that behavior. Gateway IP is outside of allocation pool, but the test is still able to allocate it as a floating IP.

I hope that clears my intentions up a bit, but if not I'm happy to clarify further and discuss my assumptions about OpenStack's floating IP allocation approach.

[Oleg Bondarev (obondarev)]

What about "reserved" or "Gateway" pre-created/placeholder port as a workaround? It will guard against Floating IP creation with same IP

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Hello:

I didn't understood the problem until the last messages. We do care about the GW IP, as confirmed by Lajos in [1]. The subnet allocation pools will always skip the GW IP. This is correct and the expected behaviour.

What is wrong is that we can create a FIP on the external network, defining manually the IP address. The server will enforce **only** that the FIP IP address is inside the subnet CIDR, **without** enforcing the limitations of the allocation pools.

E.g.: https://paste.opendev.org/show/812603/

So yes, this is a legit error.

Regards.

[1]https://docs.openstack.org/api-ref/network/v2/index.html?expanded=create-subnet-detail#id279

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

Just to amend my comment: if the FIP is requested without a manually defined IP, the assigned one will be inside the allocation pools. In this case we enforce this limitation.

[Oleg Bondarev (obondarev)]

Need to find definition of "allocation pool" in neutron (I didn't find explicit one). In my understanding allocation pools are to control _dynamic_ IP address allocation: manual IP assignment out of allocation pool was always possible - so we have to find a clear justification in order to forbid it.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

You are right, Oleg. I need to dig more on this, check the IPAM module and what an allocation pool is for and not.

[Rodolfo Alonso (rodolfo-alonso-hernandez)]

I confirm what Oleg commented: a subnet allocation pool limits the dynamic IP allocation. If a port is created without a defined IP, the IPAM module will limit the IP assignation to the allocation pool range (or ranges, a subnet can have multiple pools).

When a port is defined with an IP address, the only limitations are the existing IP addresses and the subnet CIDR.

[Sebastian Lohff (sebageek)]

I agree that using the allocation pool is the wrong solution to this problem. The problem itself is still a valid problem in my opinion and here is why:

External networks are managed by admins. The gateway ip of a subnet of an external network is normally managed outside of OpenStack (preconfigured by an admin on a router or whatever), so no Neutron object exists for the gateway ip except this one value. It is still used as next-hop for all the l3 routing. Now comes along a non-admin user that creates a floating ip using this specific external network gateway ip. As there is nothing to prevent this, Neutron will allow this FIP to be created, resulting in a second entity ARPing for this ip address. The gateway will not (or only intermittently) be reachable, as two entities ARP for the same IP. The network will not be usable for Internet access for anyone using it, including the user, who allocated the gateway IP as a FIP.

Therefore I think it makes sense to prevent users to allocate FIPs from external networks, which are also a gateway ip of the external network.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/904783