[rfe] network router:external field not exported

[Maurice Escher]

Hi, I want to use the network RBAC Feature to give 'access_as_external' to a target project so that this project is able to allocate floating IPs from that external network (without being owner or admin).

Let's say the external network has two subnets.

So far so good, but if I want to select one subnet on floating ip allocation, that is not possible. I don't see that subnet (I see the IDs, but this is useless for the decision what subnet to choose the floating ip from) and I can't give access in the policy based on the 'router:external' field of the network.

[Armando Migliaccio (armando-migliaccio)]

Do you mean that the external network has multiple floating IP pools (as subnets) associated to the external network?

[Armando Migliaccio (armando-migliaccio)]

Assigned to Kevin Benton for initial triage.

[Kevin Benton (kevinbenton)]

I'm not quite I understand the request here. Doesn't the same problem of not being able to select a specific subnet from which to allocate apply to an external network available to everyone?

I'm also having a little problem understanding how the title "network router:external field not exported" relates to floating IP allocation. Can you clarify?

[Maurice Escher (maurice-escher)]

The floating IP allocation is just an example to illustrate the usecase.

And yes, this applies also to an external network that is available to everyone. I believe rbac to everyone (read as rbac to multiple target projects) is just a special case of rbac to one target project.

I think it is missing here https://github.com/openstack/neutron/blob/d26b96b7a27c8fcd5e99a6494de406436a48339e/neutron/objects/network.py#L183

Basically I want to be able to write a policy rule like https://github.com/openstack/neutron/blob/d26b96b7a27c8fcd5e99a6494de406436a48339e/etc/policy.json#L14 also on subnets, but I can't access that network field there, so it won't be evaluated.

Or maybe this isn't even the right approach to solve this? Maybe the subnet should somehow inherit the RBAC from the network?

[Kevin Benton (kevinbenton)]

So you want to be able to expose a subnet conditional on the parent network being exposed?

[Maurice Escher (maurice-escher)]

Yes, either expose it implicitly via the parent network, which would be good imho or what also could work would be the possibility to expose it explicitly.
For the latter I don't see a use-case right now, but this would give more control.

[Kevin Benton (kevinbenton)]

So the policy engine can do lookups of parent resources. I need to dig in and see what we need to do to enable that type of policy to be written. Then we can have a separate discussion of if we should expose subnets by default.

[Kevin Benton (kevinbenton)]

POC here we can discuss next meeting: https://review.openstack.org/476094

[Kevin Benton (kevinbenton)]

ok, so I am going to adjust the patch to make the change to the policy engine to allow this rule, but it won't be the default for now so you will need to edit /etc/neutron/policy.json in your deployment.

[Armando Migliaccio (armando-migliaccio)]

We talked about this request during the latest drivers meeting. It turns out now that some prototyping has been made and more thought has been given, this request may turn out to be more hairy than it initially appeared.

Can you elaborate on what exactly you are trying to achieve? There may be potential alternatives to that that we'll help us avoid potential regressions or cause for confusion.

[1] http://eavesdrop.openstack.org/meetings/neutron_drivers/2017/neutron_drivers.2017-07-20-22.00.log.html

[Maurice Escher (maurice-escher)]

Currently I use the RBAC Feature 'access_as_shared' on external networks, so I can list the subnet via the policy "field:networks:shared=True".

But that is only a bad workaround, because by this the external networks can also be used to boot an instance, what I want to disallow eventually (because it doesn't work in our setup) and using 'access_as_external' instead.

With this fix allowing to specify in the policy "field:networks:router:external=True" to control subnets would make this clean.

[Armando Migliaccio (armando-migliaccio)]

Maurice, thanks for the extra feedback. I think what's still not clear is the workflow that the involves the selection of a subnet within an external network based on a specific CIDR. Multiple external networks and network tagging can potentially address the same need without the tight coupling of subnet selection and IP range, which doesn't seem very portable.

Can you elaborate a bit further?

[Maurice Escher (maurice-escher)]

We extend our networks by adding additional subnets. Once one range is full we can add another, where needed - and it must be possible to choose the new one/that one that has IPs available.

Additionally different subnets can differ in routing (internet vs. internal) or may need different firewall openings.

[Armando Migliaccio (armando-migliaccio)]

I can see why one would want to choose one subnet over another because of different policies (routing or security related ones), but a tenant shouldn't be concerned about IP exhaustion because neutron will select an IP automatically from a range with available IPs, and I can't see this last one being a valid use case.

On the other end, I wonder if it's really good practice to add multiple IP spaces to the same L2 domain. Though it's doable, I personally feel it being an edge case, and certainly one where it should not be used to select certain IP spaces based on high-level criteria like the ones you mentioned.

That said, I appreciate that today the ability for a low-priv user to create a floating IP on a given subnet is crippled by the fact that the tenant doesn't know about the subnet and has not way of knowing due to the lack of access to the resource when associated to an external network.

We'll continue the discussion at the drivers meeting today.

[Kevin Benton (kevinbenton)]

@Maurice,

Can't the different routing policies be expressed by exposing them as entirely separate external networks?

[Maurice Escher (maurice-escher)]

No, because we would need two routers and two internal networks. And then internal traffic would need to be natted, not an option.

[Armando Migliaccio (armando-migliaccio)]

Two routers can be connected to the same internal network (subnet), it's just a different host route. I am not sure I understand what you mean by internal traffic being natted, as I don't think it would.

[Maurice Escher (maurice-escher)]

Theoretically with the API it is possible yes, but practically we believe that two routers on the same internal network is a problem in general. Definitely it doesn't work with our hardware.

[Armando Migliaccio (armando-migliaccio)]

After some consideration, one way to tackle the use case you have in mind is to expose a new API that returns the mapping (external network UUID <-> list of SUBNET CIDRs), which is somewhat equivalent with what nova used to have with nova-network (CLI: nova floating-ip-pool-list), whose only reference I could find is [1]. That way, you can first find out which pool is suitable for your purposes and then go and uplink the router.

[1] https://www.mirantis.com/blog/configuring-floating-ip-addresses-networking-openstack-public-private-clouds/

[Maurice Escher (maurice-escher)]

Thanks, that would work I hope.

Though I don't understand why showing a subset of subnet attributes (id+cidr and maybe name+description) via a new API is supposed to be easier than showing the whole subnet somehow with the existing API.

[Armando Migliaccio (armando-migliaccio)]

It's a matter of preserving backward compatibility, and preventing leaking of infrastructure details to tenant users. I think the new API would still be admin only, but policy customization would allow to relax that.

[Armando Migliaccio (armando-migliaccio)]

We settled with proposal made at comment #19. Adding a new API is relatively straightforward, as in this case it requires no data model changes. It touches both client and server side and it would be a great way for someone to learn the ropes. We should approve this one and allow volunteers to pick it up.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/476094
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/555088

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/556674

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/556674
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=95e72ea7177aa53326a0e6c650cddcad3b4c9526
Submitter: Zuul
Branch: master

commit 95e72ea7177aa53326a0e6c650cddcad3b4c9526
Author: Hongbin Lu <email address hidden>
Date: Mon Mar 26 22:34:03 2018 +0000

    Add the floatingip pools extension

    Neutron patch: https://review.openstack.org/#/c/555088/

    Change-Id: I7bcd5350b7648008182fa509bb4c93a7c6cf5fca
    Related-Bug: #1653932

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/584502

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/555088
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4e3fb3191927e07b37fe0937102dc17334e906d1
Submitter: Zuul
Branch: master

commit 4e3fb3191927e07b37fe0937102dc17334e906d1
Author: Hongbin Lu <email address hidden>
Date: Wed Mar 21 22:39:24 2018 +0000

    Introduce floating IP pool resource

    Add support for listing floating ip pools (subnets).
    A new API resource ``floatingip-pools`` is introduced.
    This API endpoint can return a list floating ip pools
    which are essentially mappings between network UUIDs and
    subnet CIDRs. Users can use this API to find out the pool
    to create the floating IPs.

    Related patches:
    * neutron-lib: https://review.openstack.org/#/c/556674/
    * tempest-plugin: https://review.openstack.org/#/c/562038/

    APIImpact add floatingip pools api
    Change-Id: Iaa995630645042520df67d95271e14f11ffcff8c
    Partial-Bug: #1653932

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/584502
Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=e0d27124f1abb12c08d366985bd4c19d63707636
Submitter: Zuul
Branch: master

commit e0d27124f1abb12c08d366985bd4c19d63707636
Author: Hongbin Lu <email address hidden>
Date: Fri Jul 20 19:40:31 2018 +0000

    api-ref: document floating ip pools endpoint

    Neutron patch: https://review.openstack.org/#/c/555088/

    Depends-On: Iaa995630645042520df67d95271e14f11ffcff8c
    Change-Id: I88110de8d9d4151b8e8e14172b5dcd4e804eee1e
    Related-Bug: #1653932

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/562038
Committed: https://git.openstack.org/cgit/openstack/neutron-tempest-plugin/commit/?id=db2e6c90f9085368c0bda5428051c6901ac435b8
Submitter: Zuul
Branch: master

commit db2e6c90f9085368c0bda5428051c6901ac435b8
Author: Hongbin Lu <email address hidden>
Date: Tue Apr 17 20:15:35 2018 +0000

    Test floatingip pools service plugin

    Add a test case to list floating IP pools and create a floating
    IP from a pool. Assert the floating IP is created successfully.

    Neutron patch: Iaa995630645042520df67d95271e14f11ffcff8c

    Change-Id: I4f269c2cf5e3f67b766d9ccb45fd145640de68f3
    Related-Bug: #1653932