Floating IP association without subnet gateway IP

[Zsolt Krenák]

Hi All!

There is a limitation in neutron/db/l3_db.py code, that not allows associating floatingip on a subnet where the subnet gateway ip in the database is not set, but the router has a port and ip on that subnet. This limitation excludes usecases where the user do not want to set default gateway on the subnet where the floating ip is associated, but use special host routes instead.

A real life use case:
An instance got two networks, one customer facing provider vlan with public internet address, and one GRE based tenant network for operations and maintenance. The the external network providing floating ips is from the company's intranet (like 10.0.0.0/8). The default gateway obviously should be pointing towards the public interface and special host routes used for private address ranges, for example 10.0.0.0/8 used for the intranet communication. In this usecase the user need to assign floating ips on a network where the default gateway is not set to reach the instance from the company's network.

Code part causing the problem:

def _get_router_for_floatingip(self, context, internal_port,internal_subnet_id,external_network_id):
      subnet = self._core_plugin.get_subnet(context, internal_subnet_id)

      if not subnet['gateway_ip']:
           msg = (_('Cannot add floating IP to port on subnet %s '
                            'which has no gateway_ip') % internal_subnet_id)
      raise n_exc.BadRequest(resource='floatingip', msg=msg)

Simply commenting out the validation on subnet['gateway_ip'] allows the user to associate floating ip on networks where no default gateway is set, and use without any kind of problem. Of course the virtual router still has to have port on that subnet.

[Hong Hui Xiao (xiaohhui)]

If a subnet don't provide a gateway ip, it can't plug to a virtual router. This subnet will not have L3 service, so forbiding floatingip to be associated with subnet without gateway ip, looks reasonable for me.

[Hong Hui Xiao (xiaohhui)]

Oh, you mean manually adding a port in the subnet to virtual router as a gateway. If proper route is added, this might be a feasible way.
Revert the status of this bug, and leave others to comment.

[Zsolt Krenák (zsolt-krenak)]

Exactly. It is may be a little advanced usage to tinker with special routes, but I think the possibility should be there. May be some kind of warning cloud be proper when adding a floating ip without default gateway.

[Miguel Angel Ajo (mangelajo)]

@xiaohhui, I think you can plug a route in a subnet with no gateway_ip , I don't know how is that exactly handled.

To my understanding 'gateway_ip' specifies a preference for the gateway ip address... but I may need to check those facts, I always create subnets without gateway_ip and that works for me.

[Miguel Angel Ajo (mangelajo)]

is subnet['gateway_ip'] updated when we plug a router port into it?.

[Hong Hui Xiao (xiaohhui)]

@mangelajo, is subnet['gateway_ip'] updated when we plug a router port into it?
No, I don't think so.

@zsolt-krenak
Currently, there is a warning, isn't it proper?
"Bad floatingip request: Cannot add floating IP to port on subnet 43e3542b-f463-440e-ab93-64ba47cc9194 which has no gateway_ip."

[Zsolt Krenák (zsolt-krenak)]

I wouldn't consider it a warning, it's an error message. When you get this your request is denied. If you would get a warning, it would mean that the request is processed but be aware that it may not necessarily work without proper config. By the way, I haven't seen this kind of warning mechanism in any openstack project yet... Either is error or nothing.

[Carl Baldwin (carl-baldwin)]

The purposes of the gateway_ip on the subnet are:

1. Provide a default address for a router port to take when plugging a router in to the subnet. A router port can be explicitly set to use any available IP on the subnet whether or not the subnet gateway is set.
2. DHCP delivers this IP address as the gateway address to each VM instance as part of the DHCP lease.
3. When scheduling a floating IP to a router, if there are multiple routers plugged in to the private subnet, the one with the gateway will be preferred. If none of the routers have the gateway IP address then one will be chosen arbitrarily. However, the code you mention seems to prevent the latter case from happening at all.

So, if you're managing host routes out of band, and you are aware of the limitation in 3 then I guess your proposed solution could work. I'm just a bit nervous that there are some strange corner cases here which could be really confusing.

Have you explored solving this outside of Neutron?

[Zsolt Krenák (zsolt-krenak)]

Hi! Sorry for late answer, at the moment we don't see any solution for this outside neutron.

[Launchpad Janitor (janitor)]

[Expired for neutron because there has been no activity for 60 days.]

[Ross Krumbeck (rkrum)]

As a multi-tenant cloud provider we have the requirement of separating public VM traffic and backup VM traffic to our backup servers on our physical network whilst also isolating traffic between different clients.

We are using DVR and our compute nodes have different physical interfaces for public VM traffic and the backup VM traffic.

We are using vlxan for VM networks and have the following setup:

public_external_subnet (vlan provider) mapped to physnet1 (public VM interface)
backup_external_subnet (vlan provider) mapped to physnet2 (backup VM interface)

Then for the project/tenant:

client_public_subnet attached to client_public_router
client_backup_subnet attached to client_backup_router

And then we have an instance with:

eth0 connected to client_public_router with an ip from client_public_subnet on eth0 with a floating ip from public_external_subnet
eth1 connected to client_backup_router with an ip from client_backup_subnet on eth1 with a floating ip from backup_external_subnet

So what we want to do is have the default route go out eth0 and then have a static route specifically for our backup server network go out eth1.

But this is not possible due to the "if not subnet['gateway_ip']" code when adding the floating ip on the for the instance backup interface. When setting the gateway on the private subnet, this will result in the default gateway being updated for the instance public traffic to be incorrectly routed out the private interface.

[Matthew Taylor (matthew-taylor-f)]

I tested this out based upon the comments above, and I was able to achieve what I needed.

What I did after patching l3_db.py:

I added a new network (ie. 10.1.2.0/24), then created a gateway for it (10.1.2.1, network:router_interface_distributed). Later, I ticked 'Disable Gateway' for this particular network. The router_interface_distributed remained in an UP state.

I assigned a static route on then newly created network, pointing to the router_interface_distributed's gateway IP as listed above.

I assigned a new interface to the VM (eth1 in my case), which used an IP from the /24 above.

I assigned a floating IP to the VM pointing to the IP which was assigned to eth1.

After everything above, I still maintained my existing default route on eth0, and no default route on eth1. Traffic between the floating IP <---> remote end which I added a static route for works perfectly fine.

I don't see any reason as to why this shouldn't go ahead, rather than being wishlisted. Perhaps instead of blocking the action of assigning a floating IP to a network without a gateway, you should display a notification/warning instead and let the action proceed.

[Marc Heckmann (marc-w-heckmann)]

The use case as described by the original poster is legitimate and we have the same use case.

I only see two possible solutions:

1. Allow users to add a floating IP even if the gateway is not set (as the original poster suggests). Maybe with a Warning. After all, there are plenty of other ways that a user can create non-functional self-service networks in Neutron (e.g. playing with host-routes in subnets or routers)

OR

2. Provide a separate attribute for the subnet that allows us to turn off DHCP distribution of the default route while still preserving a valid IP in the "gateway_ip" field of the subnet.

[Matthew Taylor (matthew-taylor-f)]

I think that a warning is quite ideal.

[Matthew Taylor (matthew-taylor-f)]

Unassign for now, I have it in my 'to-do' list locally, and will pickup if I get the chance in the future.

[Durga Malleswari Varanasi (durga1)]

Hi Matthew,
 I am interested in working on this issue. Could you please confirm whether you are working on it currently?

[Matthew Taylor (matthew-taylor-f)]

Go ahead. :)

[Durga Malleswari Varanasi (durga1)]

Hi Matthew,

     I have gone through the patch provided. I have tried to attach a Floating Ip to an instance of a subnet where the gateway_ip is not set. And I have tried to display a warning as follows:

Log Captured:

2017-02-20 04:23:07.320 DEBUG neutron.api.v2.base [req-c350076b-c474-4d7d-aff6-af79802f6d67 admin 91476e91e8c7446bb1ef4c58daf9a538] Re
oatingip': {u'port_id': u'78316bf9-a905-4fc7-815f-e4a3d1d06400'}} from (pid=28291) prepare_request_body /opt/stack/neutron/neutron/api
2017-02-20 04:23:07.470 WARNING neutron.db.l3_db [req-c350076b-c474-4d7d-aff6-af79802f6d67 admin 91476e91e8c7446bb1ef4c58daf9a538] You are Trying to add floatingip to subnet 5f55f825-9fc0-4c7b-a165-e4ad91c663f9 which has no gateway_ip
2017-02-20 04:23:07.524 DEBUG neutron.callbacks.manager [req-c350076b-c474-4d7d-aff6-af79802f6d67 admin 91476e91e8c7446bb1ef4c58daf9a5

Could you confirm whether it is fine to display the warning like this.

[Matthew Taylor (matthew-taylor-f)]

Thanks for that Durga,

While an warning in the log files is OK for me (personally), I think you would need to cater for Horizon too.

[Durga Malleswari Varanasi (durga1)]

Hi Zsolt Krenak, Mathew Taylor

   As part of my analysis on this issue, the scenario is the limitation of the current neutron code regarding raising an exception whenever a floatingip is attached to an instance whose subnet's gateway-ip is not connected to the current router, but the router has an interface in that subnet. I am attaching a Patch regarding the issue, which contains the implementation of:

1) Throwing a Warning in the Logs in neutron server regarding the scenario

2) Displaying a Warning message in the horizon conditionally

   So that we are able to to attach a floatingip to an instance whose subnet is connected to the router(otherthan the gateway-ip of subnet) with a warning message which is not harmful. Could you please confirm whether the patch meets our need or need to have some changes.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/451665

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/451665
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=d5556a5e4eae3e638c4006987c7a334056b700f4
Submitter: Jenkins
Branch: master

commit d5556a5e4eae3e638c4006987c7a334056b700f4
Author: durga.malleswari <email address hidden>
Date: Thu Mar 30 11:53:08 2017 +0530

    Floating IP association without subnet gateway IP

    Neutron throws an exception when we try to attach the floating ip
    to the instances that are launched in a network whose subnet is
    connected to the external router by an Ip address otherthan its
    gateway-ip. Now the neutron will execute the request with no
    exception/warning.

    Closes-Bug: #1515990

    Change-Id: If212c36d918ed57400a53f4b5fa1925b3d1fa6fd

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.0.0b2 development milestone.

[Akihiro Motoki (amotoki)]

There is nothing to do in horizon side as the neutron API wrapper in the horizon code does not depend on gateway_ip of a subnet. Marking this as Invalid in horizon side.

Note that the neutron side fix which already landed is enough to fix this bug.