[RFE] No way to differentiate beween floating IP networks and external provider networks

[Sam Morrison]

We have a bunch of external shared provider networks that users can attach a port to and get direct access to the Internet.

We also have a bunch of floating IP networks that users can use for floating IPs.

The two types of networks are shared and external.

The issue is that our users can't develop code against our cloud to figure out what network to use for floating IPs.

Luckily for us our provider networks use a different provider:network_type to our floating IP networks so they can do:

search_opts = {'provider:network_type': 'midonet', 'router:external': True}
client.list_networks(**search_opts).get('networks')

But this is not very nice and by no means portable with other openstack clouds they use.

I think https://bugs.launchpad.net/neutron/+bug/1721305 is related to this bug

[Gary Kotton (garyk)]

You are able to leverage the tag feature that we have in neutron here you can take a network and then when your developers do a network query they can make use of the tag.
Will this address the issue?

[Gary Kotton (garyk)]

For example:

1. Admin user:

nicira@utu1604template:~$ openstack network set --tag 'neutron rocks' public
nicira@utu1604template:~$ neutron net-show public
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | defaultv3 |
| created_at | 2018-01-14T07:47:20Z |
| description | |
| dns_domain | |
| id | 354cbac7-3886-4931-9e1c-b26e9bff0fab |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False |
| name | public |
| port_security_enabled | True |
| project_id | ace8a4bfe20848439f403f72329c0092 |
| provider:network_type | l3_ext |
| provider:physical_network | 00f71468-d993-419b-988c-faa34fbed472 |
| provider:segmentation_id | 0 |
| revision_number | 6 |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | ac25bb28-3020-477e-8c47-3aa97b679a56 |
| tags | neutron rocks |
| tenant_id | ace8a4bfe20848439f403f72329c0092 |
| updated_at | 2018-01-16T06:17:34Z |
+---------------------------+--------------------------------------+
nicira@utu1604template:~$

2. Tenant user:
nicira@utu1604template:~$ neutron net-show public
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | defaultv3 |
| created_at | 2018-01-14T07:47:20Z |
| description | |
| dns_domain | |
| id | 354cbac7-3886-4931-9e1c-b26e9bff0fab |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False ...

[Nuwan (nuwan-ag)]

Unfortunately, I don't think it will - the problem is that we need to have a way to identify an external floating ip network that is compatible with any OpenStack installation, not only this one.

For example, in the OpenStack SDK, the create_ip function explicitly requires a floating_network_id as follows:
os_conn.network.create_ip(floating_network_id=<network_id>)

Where floating_network_id is a valid network id. However, if there's no generic, cross-openstack way to filter valid floating_network_ids, then there's no way to reliably create and assign a floating ip to an instance on an arbitrary OpenStack setup? If the proposed tag were standardised and guaranteed to be available though, sort of like 'router:external', that could be a potential solution.

[Nuwan (nuwan-ag)]

I'd also like to propose that this ticket be elevated to a higher priority, because the implication is that OpenStack doesn't have a reliable way to assign a floating ip to an instance - which is essential functionality.

Currently, we've managed to work around this with an admittedly unholy fix (https://github.com/gvlproject/cloudbridge/commit/2f049b8e4555525b5f5c15c0589ee501c3189688#diff-a668445303da4930cbf7f86928ae2c77R731), which is to simply test every available external network as a potential floating ip network by attaching it to a gateway, which is very clunky and slow, but it is generic and cross-openstack compatible.

[Gary Kotton (garyk)]

What do you mean that OpenStack is unable to assign a floating IP?
What I am saying above is you can leverage tags to add extra properties and then filter according to that. That is provided in vanilla OpenStack. I think that it is since N or O.

[Nuwan (nuwan-ag)]

If I understood you correctly, if we have to tag the floating network in advance with a special tag, it will be difficult to write client code that runs against an arbitrary OpenStack cloud, since there's no guarantee that an arbitrary OpenStack setup would have the network tagged in this way.

So the question is whether there is a standard way to programmatically determine which external networks are valid for assigning floating IPs from, in much the same way that it's possible to determine which networks are external with the router:external property.

Without a standard way to do this, the problem we are running into is that we can't programmatically determine which floating network to create the IP in so we can pass it as a parameter to: `os_conn.network.create_ip(floating_network_id=<network_id>)`, which is a required parameter.

[Armando Migliaccio (armando-migliaccio)]

This sounds like a discussion for the drivers team.

[Sam Morrison (sorrison)]

I'm pretty sure this is an API issue, developers need a way to know which networks are floating IP networks and have this work the same on any openstack cloud.

Tags won't work as they are not consistent across clouds.

There used to be a nova floatingip-pool-list api in nova-network that you could use but this isn't in neutron.

There is a command in the openstack client which I think is for nova-network too:
$ openstack floating ip pool list
Floating ip pool operations are only available for Compute v2 network.

[Armando Migliaccio (armando-migliaccio)]

We typically operated on the assumption that the 'router:external' attribute is the qualifier for identifying networks to use for floating IP ranges, but clearly use cases have demonstrated that it's not always the case. There's a similar RFE (bug 1653932) that demands the need for capturing the available floating IP pools as you describe here.

[Akihiro Motoki (amotoki)]

As a baseline, floating IP networks can be distinguished by checking networks whose 'router:external' attribute is True.
"provider network" is a network which is created by a cloud operator. "provider network" is expected a network with 'shared=True' attribute.

On the other hand, could you clarify what is "external provider network"? Can I assume "external provider network" is a network which is a "provider network" and "floating IP network" at the same time?

There might be several corner cases where "router:external" or "shared" attributes. If there are such cases, I think we can fix them one by one.

I might be missing some basic points though.....

[Sam Morrison (sorrison)]

Here is a network that can host floating IPs:
+---------------------------+----------------------------------------------------------------------------+
| Field | Value |
+---------------------------+----------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | None |
| description | None |
| dns_domain | None |
| id | e48bdd06-cc3e-46e1-b7ea-64af43c74ef8 |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | False |
| is_vlan_transparent | None |
| mtu | 0 |
| name | melbourne |
| port_security_enabled | False |
| project_id | 1 |
| provider:network_type | None |
| provider:physical_network | None |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | 3 |
| router:external | External |
| segments | None |
| shared | True |
| status | ACTIVE |
| subnets | 0f7520e4-1777-468a-9047-0a87a13e178a, b4771a97-bfb8-4e48-89af-10cf88e63a7f |
| tags | |
| updated_at | None ...

[Sam Morrison (sorrison)]

Does anyone have any ideas on how to do this, I'm worried that this going to wishlist this will never get much attention unless we have a plan.

My guess is most deployments provide only one way to do external traffic. Either by providing external provider networks OR by floating IP networks and few give their users both options. (last time I used RAX this was also supported so maybe not too uncommon?)

It would be nice if there was an API call you could do like in the nova-network days to list all the floatingip pools, maybe that is the way to go?

I guess that would just mean that a network would have a new attribute on it corresponding to whether it can be used for floating IPs or not.

[YAMAMOTO Takashi (yamamoto)]

can auto-fip [1] serve the use case?
or do your users want to develop more complex policies?

[1] https://review.openstack.org/#/c/469294/

[Akihiro Motoki (amotoki)]

If you would like to know which network can be used as floating IP pool, the recommended way is to query networks with router:external=True. Isn't it enough? I don't see any reasonable answer from the bug reported.

In OSC, you can do this:

$ openstack network list --external

OSC might add some command for user convenience like "openstack floating ip pool list", but it is a different topic from the API.

[Sam Morrison (sorrison)]

It is also possible to have a external=true network that isn't a floating IP network.

We have a provider network that is external and users can attach a port from this network directly to their instance. They can't however use this network as a floating IP network.

[Miguel Lavalle (minsel)]

@Sam,

We are going to discuss this RFE during the next Neutron drivers meeting. What is the specific solution that you propose?

[Sam Morrison (sorrison)]

OK great, I'd be happy to come along if timezones work out, when is it?

In terms of a solution I'm not sure of the best approach.

Possibly being able to set a network as being a floating IP network might work and then being able to filter networks by that attribute.

The solution needs to be something that can be consistent across all openstack clouds so this can be built into all client libraries.

[Miguel Lavalle (minsel)]

Hi Sam, we are going to hold the meeting on Firday March 16th at 1400UTC. It will take place in freenode, channel #openstack-meeting: http://eavesdrop.openstack.org/#Neutron_drivers_Meeting

[Akihiro Motoki (amotoki)]

> It is also possible to have a external=true network that isn't a floating IP network.

> We have a provider network that is external and users can attach a port from this network directly to their instance.
> They can't however use this network as a floating IP network.

I think there is some confusion on the usage of "router:external" and "shared" attributes.

- shared=True means users can attach a port from this network directly to their instance
- router:external=True means this network can be used as a floating IP network.

Looking at comment #11 (https://bugs.launchpad.net/neutron/+bug/1743480/comments/11), you set both shared=True and router:external=True for both networks.

In addition, if you have two network as floating IP pools and expose one of them to a specific set of users and the other of them to another set of users, you can use "access_as_external" of network RBAC feature.

I would like to clarify whether you have use cases more than above.

[Sam Morrison (sorrison)]

Hmm OK, so a floating IP network should have shared = False?

I will give that a go and see if that fixes it.

> - shared=True means users can attach a port from this network directly to their instance

OK understand, and having external=True also would mean this shared network has a router external to neutron, that all makes sense and is logical.

> - router:external=True means this network can be used as a floating IP network.

I think this is not quite true as if it is shared=True and external=True then it can't be used as a floating IP network, at least in our case but possibly all cases?

So a floating IP network can be defined as a network that is shared=False and external=True?

If that is the case I wonder if it could someway be easier as a developer to know this as it seems slightly confusing.

[Miguel Lavalle (minsel)]

This is what I see in my development system for my floating ip network: http://paste.openstack.org/show/709007/

[Miguel Lavalle (minsel)]

@Sam,

Did you do any further testing? Can we consider this RFE addressed?

[Sam Morrison (sorrison)]

I guess the main issue I see is that this is very confusing for someone trying to use openstack. I've been using it for 6 years and I couldn't figure out how to determine a floating IP network.

I really think we need to make this clearer for end users using the API. How are they supposed to know that to list all floating IP networks you need to filter the network list call by shared=False and external=True?

[Dr. Jens Harbott (j-harbott)]

IMO this is deployment-specific data that the deployer needs to specify out-of-band if consumers need it, like by providing a cloud.yaml that contains a list of the relevant networks.

Note that filtering by "shared=False" may not always give you the correct results, it is totally valid to deploy a single public network both for floating IPs and directly attached instances. In fact this is what the default devstack setup looks like.

[Sam Morrison (sorrison)]

If you are writing a piece of software that is portable and is aimed at working on any openstack cloud then there needs to be a better way. This needs to be discoverable via the api

[Miguel Lavalle (minsel)]

@Sam,

It is in fact discoverable through the API since that is what the "router:external=True" is for. We will update our documentation to be clearer on this point

[Sam Morrison (sorrison)]

OK it is discoverable but its not very friendly.

Old way via the CLI is:

nova floating-ip-pool-list

Now it's:

openstack network list --external --no-share

I feel if you're not familiar with openstack and neutron you would struggle to figure this out by yourself.