[RFE] Add OVN-based Neutron BGPVPN driver

[Luis Tomas Bolivar]

Continue the work on https://review.opendev.org/c/openstack/networking-bgpvpn/+/803161

[Maximilian Sesterhenn (msnatepg)]

As part of our efforts to add ovn-bgp-agent to kolla / kolla-ansibe, we have also worked on this driver to get it working on the recent neutron codebase and also done some modifications to allow external networks (including L2 as proposed in 2017890).

If you're interested in it I can offer to create a WIP change, even if it does not fit exactly for upstream I bet it could be helpful anyways.

[Luis Tomas Bolivar (ltomasbo)]

I'm not sure I completely got this. You mean you created a newer networking-bgpvpn patch to support the EVPN driver of ovn-bgp-agent? Also, how is this related to 2017890? it also allows to pass VNI information for that?

[Maximilian Sesterhenn (msnatepg)]

Yes, we created a newer patch as we had to do some modifications to be able to load it.
The OVN BGPVPN service driver adds some config options which are already defined in neutron itself and it seems that Oslo doesn't like that.
Not sure if that was the reason for the Zuul jobs to fail.

Regarding 2017890, yes, BGPVPN allows the user to choose between L2 and L3, in the abandoned patch L2 is blocked and L3 is only allowed on non-provider networks if I remember correctly.

Using our patch you can:

- Associate routers, to allow for exposing non-provider networks connected to the router using L3 routes. (this was already there)
- Associate networks, to allow for exposing provider networks either in L2 or L3.

In all cases, the VNI information gets added to the external_ids field in NB DB, to differentiate between L2 and L3 we added another field into external_ids that carries this information.

[Luis Tomas Bolivar (ltomasbo)]

That looks really promising! I thought networking-bgpvpn was not allowing you to associate provider networks to the BGPVPNs defined, nice that you is not the case (or you found a solution for it)!

Please send a patch so that we can review it. I hope we can make it appealing for upstream folks and also be merged in networking-bgpvpn!

[Maximilian Sesterhenn (msnatepg)]

You're welcome :P

I can not see a reason why it should not be possible to add provider networks to a BGPVPN.
From my point of view that was only a limitation of the service provider, not BGPVPN itself.

Right now I'm wondering if it makes sense to also unlock the remaining features of BGPVPN like RT or RD to give the user even more control about how FRR is announcing routes.

What do you think of it?

[Luis Tomas Bolivar (ltomasbo)]

Indeed, I had that in my TO DO list too, but I shifted a bit the focus from EVPN to BGP driver. So definitely makes sense yes.

In fact, now that we're trying to converge both drivers, probably the networking-bgpvpn would be needed as an API for the BGP drivers too, not only for the EVPN one. So it will be nice to have all the information and then each driver can leverage/enforce whatever makes sense.

That would be also linked to https://bugs.launchpad.net/ovn-bgp-agent/+bug/2018608 I suppose, right?

[Maximilian Sesterhenn (msnatepg)]

I think allowing users to override the config snippets would be beneficial in any case.

I will remove the limitations so that all BGPVPN fields can be used and will be added to external_ids. Are you aware of any limitations in terms of field length of external_ids?

The only limitation of the service provider I would keep is that networks associations can only be used with external networks, with any type (L2/L3).
Internal networks are associated implicitly through the association of the router to the BGPVPN, also any type.
Would that be alright?

Whats the reason for the BGP AS number in the service provider?
In an EVPN environment thats normally nothing thats configured on an VNI level and therefore I would expect this setting in the agent config, not in the config of the service provider.
Makes no sense to me to have this in the external_ids, but maybe you can give me a hint :)

[Luis Tomas Bolivar (ltomasbo)]

In the previous networking-bgpvpn we allowed associating networks or routers, so that you could pick what networks to expose from the ones connected to a router. But I think, given that you cannot associate 2 networks on the same router to different BGPVPNs, the limitation makes a lot of sense, so I agree with your proposal

Regarding the AS number, that was to give more flexibility to the admins defining new/different ones independently of the agent deployment, and that was used for the frr VRF configuration, but I suppose we can also assume the same ASN will be used for all VRFS, right?

[Maximilian Sesterhenn (msnatepg)]

https://review.opendev.org/c/openstack/networking-bgpvpn/+/883060

I've published a WIP change in Gerrit that contains our modifications.
It's still not finished yet as I have to do more testing and maybe adding some additional unit tests, but it should work fine already.

Regarding the AS numbers:

To my understanding networking-bgpvpn does not allow to specify an AS number for a specific BGPVPN.
Right now that value is loaded from a config in networking-bgpvpn, so up to my understanding its the same for the whole region where this instance of neutron-server is used.
Ignoring that value and using something configured on the agents would at least make this significant for only the node where the agent runs.
I think that's also what happens today, the locally configured AS is used for configuring the config snippets into FRR.
As each VNI is usually announced through the same EVPN sessions in the underlay that makes the most sense.

The only useful scenario I could imagine would be a feature where we would offer regular BGP sessions to external routers so that a given customer could connect their external networks to their cloud environment. These routes would then be installed as type 5 routes into the customers L3 VNI. But thats a different topic and to my understanding not in scope for ovn-bgp-agent and for networking-bgpvpn as well.

As to my understanding the AS number is not used and there is no usecase to define that centrally I have removed that field in the change mentioned above.

[Dan Sneddon (dsneddon)]

I think we may want to consider whether BGP Views make sense here. That is a feature where a single FRR BGPd process may have two or more torally separate BGP configurations, ASNs, peers, etc. There is no communication or visibility between views, so this is a different use case from VRFs. This is more like virtual router segregation or separation. I know I’ve brought this up before and we haven’t found a compelling use case, but hear me out.

We could consider different views for local Neutron/OVN and EVPN, perhaps with the BGP agent taking to both virtual routers.

Another use case is when the user/operator wants to run a custom BGP config on the same BGPd we are using for OVN. This may not be appropriate for every user/operator case, but it does clear up a number of issues such as configuration conflicts, potential user-created security weakness, and difficulty troubleshooting a BGP router with both automated and manual config with potential functionally overridden. Of course it does this at the cost of complexity, the user/operator must configure every aspect of the BGPd configuration and may need to make network topology changes to support multiple virtual routers on one host.

I don’t mean to imply that BGP views are a global solution, but since I don’t have all the answers I wanted to think about edge cases and think big. I believe we are on the cusp of having BGP be a critical and common part of the modern datacenter, so we don’t want to limit BGP use cases on a given host to our limited and opinionated solution. Multiple FRR/BGPd processes is not a good or scalable solution, so I propose we model BGP views to see what it gets us.

[Maximilian Sesterhenn (msnatepg)]

Hey there,

not sure if this is something that we would care about in the BGPVPN driver, as in my opinion we would add the data in the same way to the OVN DB anyway.

For me, this seems to be more a topic for ovn-bgp-agent in general, maybe focused on the FRR configuration.

I'm also working on a change in kolla / kolla-ansible (thats not public yet) where we add support for FRR and ovn-bgp-agent. Regarding the FRR deployment, we made a design choice to only include FRR configuration examples, it's still up to the admin to provide the FRR configuration for their specific environment.
So all functionality that FRR supports could be used.

Of course, as of today, this config would have to be structured in a certain way to allow the config snippets that ovn-bgp-agent injects into FRR to work correctly.

That would be something that [1] could solve.
Maybe that's the right place for this feature.

[1] https://bugs.launchpad.net/ovn-bgp-agent/+bug/2018608