[RFE] Add 'VNI' column in bgpvpns table in Neutron database.

[Siddanagouda Khot]

Current bgpvpns table in Neutron maintains VPN information. Adding a new column 'VNI' to cater to EVPN/VxLAN requirements.

=====================================================
Add ‘VNI’ column in bgpvpns table in Neutron database
=====================================================

Problem Description
===================

[1] describes the OpenStack BGP EVPN route advertisement solution. This solution uses BGPVPN [2] to store and retrieve VPN information. Current bgpvpns table in BGPVPN doesn’t have VNI column. Adding ‘VNI’ column addresses the EVPN/VxLAN [3] reqiurement.

Proposed Change
===============
Add ‘VNI’ column to bgpvpn table in Neutron database which is also mentioned in [2]

Data Model Impact
-----------------

Current bgpvpns table has following fields:
 - name
 - id
 - tenant_id
 - type
 - route_targets
 - import_targets
 - export_targets
 - route_distinguishers

A new ‘vni’ filed will be added:
vni = sa.Column(sa.Integer)

Respective CRUD functions update:

class BGPVPNPluginDb(common_db_mixin.CommonDbMixin):
                 route_targets=rt,
                 import_targets=i_rt,
                 export_targets=e_rt,
- route_distinguishers=rd
+ route_distinguishers=rd,
+ vni=bgpvpn['vni']
             )

References
==========

[1] OpenStack BGP EVPN IP Prefix advertisement: http://docs-draft.openstack.org/54/322654/2/check/gate-neutron-specs-docs/e06688e//doc/build/html/specs/newton/bgp-evpn-advertisement.html

[2] BGPVPN: http://docs.openstack.org/developer/networking-bgpvpn/

[3] A NVO solution for EVPN draft: https://tools.ietf.org/html/draft-ietf-bess-evpn-overlay-03

Proposed fix: https://review.openstack.org/332711

[Siddanagouda Khot (siddanmk)]

This enhancement is required for the Opebstack BGP EVPN solution effort mentioned in http://docs-draft.openstack.org/54/322654/2/check/gate-neutron-specs-docs/e06688e//doc/build/html/specs/newton/bgp-evpn-advertisement.html

[Thomas Morin (tmmorin-orange)]

Hi Sid,

Most of the work on adding VNID attribute to the API was done when we did the initial work on the API.
It was later moved off the API doc, because nothing was implementing it, but the documentation was preserved at:

http://docs.openstack.org/developer/networking-bgpvpn/future/attributes.html#vnid-attribute

(The name chosen at the time was 'vnid', but 'vni' is what VXLAN spec use, so we can change that.)

Please review this text, the semantic of the vni field required some attention to detail and you need to check if this matches what the bgp-evpn-advertisement spec needs.

[Mickey Spiegel (emspiege)]

Thomas,

Thanks for the link to
http://docs.openstack.org/developer/networking-bgpvpn/future/attributes.html#vnid-attribute

I agree with the first half. I find the text for a network or router associated with multiple BGPVPNs confusing. The only way I can make sense of it is to change the second bullet to read:
- the route is used to establish connectivity to the destination in the forwarding plane only if the received VNID is contained in the set of retained VNIDs in the previous step.

For received routes, I guess the question is whether to filter the received VNI within each BGPVPN independently, or filter the received VNI across the set of BGPVPNs associated with the network or router.
For example, if BGPVPN 1 has import RT 1 and VNI 1, and BGPVPN 2 has import RT 2 and VNI 2, should a received route with RT 1 and VNI 2 be accepted?
I think the answer should be no. In our backend design, we intend to do the processing on a per VNI basis.

My proposed alternative:

when a route is imported, for each BGPVPN associated to the Network or Router and having a VNID defined:
    the set of Route Targets of the route is intersected with the import_rts of the BGPVPN
    if this intersection is non-empty, and the ‘VNID’ of the BGPVPN is equal to the VNID defined for that BGPVPN, then the route is used to establish connectivity to the destination in the forwarding plane

Regarding advertisement, if the VNIs vary over the multiple BGPVPNs, then the prefix(es) will have to be advertised separately for each VNI. I guess a similar issue exists with respect to RDs, requiring separate advertisements if the RDs are different. The RTs for each prefix advertisement will be the union of the export RTs for all associated BGPVPNs with that VNI value (and RD value).

[Thomas Morin (tmmorin-orange)]

Mickey,

> I find the text for a network or router associated with multiple
> BGPVPNs confusing. The only way I can make sense of it is to change the second bullet to read:
> - the route is used to establish connectivity to the destination in the forwarding plane only if the
> received VNID is contained in the set of retained VNIDs in the previous step.

The current formulation may need to be improved, but your reformulation does not match our initial intent.

The intent was to say that a route can be used only if the VNI in the route matches the VNI of all the BGPVPNs that import it (if there is one BGPVPN that imports the route and has a VNI different from the one of the route, the route can't be used, because using it would result in choosing a VNI that would violate whether the VNI specified in the route or the VNI specified by that BGPVPN).

> For received routes, I guess the question is whether to filter the received VNI within each BGPVPN
> independently, or filter the received VNI across the set of BGPVPNs associated with the network or router.
> For example, if BGPVPN 1 has import RT 1 and VNI 1, and BGPVPN 2 has import RT 2 and VNI 2, should a
> received route with RT 1 and VNI 2 be accepted?
> I think the answer should be no.

I agree with you.

> My proposed alternative:
>
> when a route is imported, for each BGPVPN associated to the Network or Router and having a VNID defined:
> the set of Route Targets of the route is intersected with the import_rts of the BGPVPN
> if this intersection is non-empty, and the ‘VNID’ of the BGPVPN is equal to the VNID defined for that
> BGPVPN, then the route is used to establish connectivity to the destination in the forwarding plane

Let's apply this algorithm to the a scenario where a Network is associated to both BGPVPN1 and BGPVPN2, BGPVPN1 has import RT 1 and VNI 1, BGPVPN2 has import RT 2 and VNI 2, for a route carrying both RT1 and RT2 and has VNI 1. The algorithm would accept the route based on the check applied for BGPVPN1. The fact that the check fails for BGPVPN2 will not prevent the route from being used and VNI1 will be used for a route imported based on a BGPVPN association saying "VNI2".

> Regarding advertisement, if the VNIs vary over the multiple BGPVPNs, then the prefix(es) will have to be
> advertised separately for each VNI. I guess a similar issue exists with respect to RDs, requiring
> separate advertisements if the RDs are different. The RTs for each prefix advertisement will be the union
> of the export RTs for all associated BGPVPNs with that VNI value (and RD value).

Yes, I agree.
This is something the specs currently don't capture properly (for both RDs and VNIs).
(Note that for RDs, things would be slightly different since we allow lists: a same route can be used to
advertise to two different VPNs as long as the BGPVPN don't specify non-intersecting lists of RDs.)

[Mickey Spiegel (emspiege)]

Thomas,

Thanks for the clarification of intent.

I do not get why you think of an association between a network or router and a BGPVPN as something exclusive, where connectivity between that network or router and everything else is limited to the definitions of that BGPVPN.

I think of an association between a network or router and a BGPVPN as allowing connectivity between that network or router and other things in that BGPVPN. That does not mean that the same network or router cannot connect to other things, outside the context of that BGPVPN.

The specific use case that I always have in mind for this is an application running in a multi-tenant data center, that requires connectivity both to the public internet for the front end, and back to an on premises customer site on the back end. By using BGPVPN for both aspects of connectivity, a router can learn which addresses are reachable over the BGPVPN leading to the on premises customer site, and which addresses are reachable over the BGPVPN leading to the internet.

>Let's apply this algorithm to the a scenario where a Network is associated
>to both BGPVPN1 and BGPVPN2, BGPVPN1 has import RT 1 and VNI 1, BGPVPN2
>has import RT 2 and VNI 2, for a route carrying both RT1 and RT2 and has
>VNI 1. The algorithm would accept the route based on the check applied for
>BGPVPN1. The fact that the check fails for BGPVPN2 will not prevent the
>route from being used and VNI1 will be used for a route imported based on
>a BGPVPN association saying "VNI2".

I do not agree that the route is 'imported based on a BGPVPN association saying "VNI2"'. The route is imported based on BGPVPN1, not based on BGPVPN2.

Look at RT processing. From RFC 4364: "Note that a particular VPN-IPv4 route is only eligible for installation in a particular VRF if there is some Route Target that is both one of the route's Route Targets and one of the VRF's Import Targets". As long as there is an intersection in the route's RTs and the import RTs, the route is eligible. The presence of additional import RTs does not affect route eligibility.

It is true that typically for the use case that I described above, routes will have either RT 1 or RT 2, but not both (in which case either proposal would work). However, I still contend that RT should be used as a filter to determine whether a route is eligible. Once a route is eligible, you use the VNI specified in the route. Just because there is an additional RT in the route, that should not preclude the route from being used.

[Thomas Morin (tmmorin-orange)]

Hi Mickey,

We have to keep in mind why the possibility to specify a VNI for a BGPVPN is being added.

Let's take a step back first. We don't have a similar to define MPLS labels to use for a BGPVPN. The design of the protocols (the RFCs you quote, and before them the base specs for MPLS) leverage the idea of locally and dynamically assigned MPLS labels. All MPLS stacks have been designed to support that.

VXLAN on the contrary was designed for globally assigned VNIs, and although it can be used with a strategy where VNIs are locally and dynamically assigned, some VXLAN implementation, in particular some hardware implementations ones cannot do that (or not efficiently enough). To use E-VPN VXLAN in contexts where you need to co-exist with such hardware, you need to ensure that for a given VPN, a single VNI will be used, and hence the BGPVPN API was augmented to control the VNI. Without this constraint, the VNI to use would not need to be globally coordinated and the API would not need to control it.

With this in mind, the question of the semantic of the VNI of a BGPVPN in the presence of routes that also specify a VNI, has no elegant answer. The approach you propose would go counter to the motivation for adding the VNIs in the API (support hardware requiring a single VNI for a said VPN instance).

Now, I'm realizing that the answer I gave on advertisements is not consistent with the above: on hardware having the said limitation, advertising the prefix twice with two VNIs would not be an option.

I'm leaning toward the idea of simply saying that associating something to two BGPVPNs specifying a different VNI is disallowed. If you are in a context where no such hardware limitation is in the picture, then you would just not specify a VNI for a said BGPVPN, and the backend will be free of dynamically/locally choosing the VNI to use for a given advertisement, and honor any VNI in a received route.

[Mickey Spiegel (emspiege)]

Thomas,

>VXLAN on the contrary was designed for globally assigned VNIs, and
>although it can be used with a strategy where VNIs are locally and
>dynamically assigned, some VXLAN implementation, in particular some
>hardware implementations ones cannot do that (or not efficiently enough).

I agree with this statement.

>To use E-VPN VXLAN in contexts where you need to co-exist with such
>hardware, you need to ensure that for a given VPN, a single VNI will be
>used, and hence the BGPVPN API was augmented to control the VNI. Without
>this constraint, the VNI to use would not need to be globally coordinated
>and the API would not need to control it.

Are there constraints?
Yes.

Is the constraint to have a single VNI?
For a single bridge table in a MAC-VRF in the context of L2 EVPN overlay, for some hardware implementations, yes.
However, there are EVPN prefix advertisement use cases where there may be more than one MAC-VRF / bridge associated with a router.

We want to support L3 EVPN prefix advertisement, with the following semantics:
- VLAN Based service, where a tenant VLAN ID gets mapped to an
  EVPN instance (EVI).
  The MAC-VRF table corresponds to only a single bridge table.
  i.e. Ethernet Tag = 0
- NVEs use VNIs as globally unique identifiers
- VXLAN encapsulation (original flavor, not VXLAN GPE)
- draft-ietf-bess-evpn-prefix-advertisement-02 case 5.4 but with some aspects of case 5.1.
  Specifically:
  - In Figure 6, IP-VRF on NVEs 1 and 2 corresponds to the Neutron router
  - I am not sure why Figure 6 did not show MAC-VRF 10 in between the
    IP-VRF instances, as shown in Figure 2 for case 5.1. The IRBs facing
    VXLAN would be in between the IP-VRF instances and the MAC-VRF 10
    instances.
  - MAC-VRF 10 corresponds to the Neutron network that connects the
    Neutron routers to upstream (physical) routers
  - IRB-1 and IRB-2 correspond to the Neutron router interface to the
    Neutron network that connects to upstream (physical) routers
  - MAC-VRFs 1, 2, and 3 correspond to tenant networks behind the Neutron
    router
  - Matching a vendor implementation, regarding the many options in
    section 5.4:
    (1) GW IP = 0
        Only route type 5 is advertised, no route type 2
    (2) The DGW implementation will use the VNI and next-hop of the
        RT-5, as well as the MAC address conveyed in the Router's
        MAC extended community
    (3) The RT-5 for SN1/24 has a GW-IP=0, DGW1 does NOT refer to RT-2
    (4) NVE1 will identify the IP-VRF for an IP-lookup based on the VNI
        and the inner MAC DA

The interesting part is in Section 5.1 (3):

    Based on the MAC-VRF10 route-target in DGW1 and DGW2, the IP
    Prefix route is also imported and SN1/24 is added to the IP-
    VRF ... pointing at the local MAC-VRF10.

The route target is associated with MAC-VRF10, yet somehow the prefix route is imported to IP-VRF.
How did DGW determine which IP-VRF?

I believe the same representation and the same issue applies on the NVE side as on the DGW side.
My answer for the NVE side: This is where the router association comes into the picture.
The router association is associating the bgpvpn's MAC-VRF with the router's IP-VRF.

...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/332711

[Thomas Morin (tmmorin-orange)]

Thanks for your inputs.

After giving some thought to this, I'm thinking we should keep the rule as-is for imported routes but *only* for L2 BGPVPNs (and hence only for Network associations), because the hardware limitation which lead us to add this restriction applies to a single bridge domain but not anymore if a bridge is in between.

This is consistent and I think does resolve the obstacle wrt. the use in BGP DR project.

For l3, the VNI specified would apply to advertised routes, and for received routes we could whether (a) apply something like the rule you suggest (use a route compatible with the VNI-specification of any BGPVPN, considering that a BGPVPN with no VNI specified would mean "accepting any VNI"), or (b) do not do any filtering at all (use any route received, and honor whatever VNI it specifies). Any reason to not choose the simplest (ie. (b)) ?

Thoughts ?

Side comments/answers to some of your questions below:
- section 5.1 of evpn-prefixes draft indeed lacks a bit of details on RT; my interpretation of the sentence you quote is that it means that the IP-VRF is importing from/to the RT used for the MACVRF10 (nothing prevents a same RT to be imported by multiple entities).
- you say "The DGW implementation will use the VNI and next-hop of the RT-5 [route]" ; I read this as "the DGW implementation can honor any VNI used, and is not limited to using a VNI associated to its MAC-VRF", which lead me to ask: why then do you need/want to use globally assigned VNIs ? everything is sooo simpler when dataplane ids are just locally assigned...

[OpenStack Infra (hudson-openstack)]

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/332711
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Thomas Morin (tmmorin-orange)]

Vivek, can you confirm whether or not you are still working on it ?

If yes, I would like to discuss this tomorrow during next IRC weekly meeting (Weekly on Tuesday at 1500 UTC in #openstack-meeting-alt), can you be there ?

[Vivekanandan Narasimhan (vivekanandan-narasimhan)]

I could not fetch cycles to work this out for the Pike release.

Since this is API change, and Pike-3 milestone is over, can we please capture this effort for Queens?

[Thomas Morin (tmmorin-orange)]

Vivek: yes, of course, since Pike feature freeze is way past us, this is for Queens.

However, even for a Queens timeframe, I would like to discuss early when/who this will happen.

I'm also interested in seeing this land, and ready to help to land the API definition very early in the cycle.

[Thomas Morin (tmmorin-orange)]

Vivek, can you attend tomorrow's IRC meeting ?

[Vivekanandan Narasimhan (vivekanandan-narasimhan)]

Deepthi and myself will be pursuing this effort forward.

We could not make it to IRC meeting due to OpenDaylight NetVirt meeting around the same.

Would like to land the API definition earlier in Queens.

[Thomas Morin (tmmorin-orange)]

Thanks Vivek for the update.

Now is the good time to start thing if we want to land in Queens.

The first piece to land is a neutron-lib change to define an API extension.
This would be similar to https://review.openstack.org/#/c/467277/ but simpler.

Then (or in parallel), https://review.openstack.org/332711 can be resurected.
(the CLI extension work will need to be done in python-neutronclient though)

[Thomas Morin (tmmorin-orange)]

Let me know if you need help on preparing the neutron-lib change.

Also, it's fine if you don't attend the IRC weekly and simply report progress here.

Thanks!

[Deepthi V V (deepthivv88)]

Proposed change in neutron-lib:
https://review.openstack.org/#/c/503954/

[Deepthi V V (deepthivv88)]

Proposed change in python-neutronclient
https://review.openstack.org/#/c/503951/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/505496

[OpenStack Infra (hudson-openstack)]

Change abandoned by Thomas Morin (<email address hidden>) on branch: master
Review: https://review.openstack.org/332711
Reason: abandoning in favor of I3d47000bb0ad8434f88d016acb5b05fb7142e47a

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/505496
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=74dedf51b865a2aaf7ba52f200c40fd716c96539
Submitter: Zuul
Branch: master

commit 74dedf51b865a2aaf7ba52f200c40fd716c96539
Author: Deepthi V V <email address hidden>
Date: Tue Sep 19 11:13:39 2017 +0530

    Add vni attribute to bgpvpn resource

    Co-Authored-By: <email address hidden>
    Co-Authored-By: <email address hidden>
    Co-Authored-By: <email address hidden>

    Change-Id: I3d47000bb0ad8434f88d016acb5b05fb7142e47a
    Partial-Bug: #1590908
    Depends-On: Iae5fe901f0cddfbf8b1ef3afd529f9131b0aa165